[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:13 dist-test-slave-fgb3 krb5kdc[5617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:13 dist-test-slave-fgb3 krb5kdc[5617](info): set up 2 sockets
Aug 07 03:02:13 dist-test-slave-fgb3 krb5kdc[5617](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 03:02:14.971526 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:15 dist-test-slave-fgb3 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:15.141898 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40435
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:40435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:15.490927 5636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:15.491323 5636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:15.491461 5636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:15.498888 5636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:15.498996 5636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:15.499068 5636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:15.499146 5636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:15.499215 5636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:15.509619 5636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:40435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:15.511816 5636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:15.513862 5636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:15.538024 5644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:15.538837 5646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:15.538918 5648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:15.539757 5636 server_base.cc:1034] running on GCE node
I20260807 03:02:15.540908 5636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:15.543418 5636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:15.544631 5636 hybrid_clock.cc:648] HybridClock initialized: now 1786071735544541 us; error 87 us; skew 500 ppm
I20260807 03:02:15.545171 5636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:15 dist-test-slave-fgb3 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071735, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:15.548714 5636 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:15.551414 5636 webserver.cc:460] Webserver started at http://127.5.122.62:43775/ using document root <none> and password file <none>
I20260807 03:02:15.552539 5636 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:15.552654 5636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:15.553083 5636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:15.557247 5636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "90c5730ac9b642dba0019d98b2fdfe2a"
format_stamp: "Formatted at 2026-08-07 03:02:15 on dist-test-slave-fgb3"
server_key: "12ce892a13376fd6945b6e071c8742ee"
server_key_iv: "9b7860724057b5a19cd9dec78fa7abbd"
server_key_version: "encryptionkey@0"
I20260807 03:02:15.558368 5636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "90c5730ac9b642dba0019d98b2fdfe2a"
format_stamp: "Formatted at 2026-08-07 03:02:15 on dist-test-slave-fgb3"
server_key: "12ce892a13376fd6945b6e071c8742ee"
server_key_iv: "9b7860724057b5a19cd9dec78fa7abbd"
server_key_version: "encryptionkey@0"
I20260807 03:02:15.566207 5636 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 03:02:15.571744 5660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:15.573572 5636 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260807 03:02:15.573827 5636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "90c5730ac9b642dba0019d98b2fdfe2a"
format_stamp: "Formatted at 2026-08-07 03:02:15 on dist-test-slave-fgb3"
server_key: "12ce892a13376fd6945b6e071c8742ee"
server_key_iv: "9b7860724057b5a19cd9dec78fa7abbd"
server_key_version: "encryptionkey@0"
I20260807 03:02:15.574018 5636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:15.590411 5636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:15.595289 5636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:15.595618 5636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:15.616261 5636 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:40435
I20260807 03:02:15.616297 5749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:40435 every 8 connection(s)
I20260807 03:02:15.618569 5636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:15.620644 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5636
I20260807 03:02:15.621052 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:15.624015 5608 external_mini_cluster.cc:1428] Setting key 38e4a300391d45fcbe71442d36ad68c4
I20260807 03:02:15.626926 5751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:15.649050 5751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a: Bootstrap starting.
I20260807 03:02:15.655818 5751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:15.657675 5751 log.cc:824] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:15.662407 5751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a: No bootstrap required, opened a new log
Aug 07 03:02:15 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:15.671728 5751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40435 } }
I20260807 03:02:15.672374 5751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:15.672533 5751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c5730ac9b642dba0019d98b2fdfe2a, State: Initialized, Role: FOLLOWER
I20260807 03:02:15.672778 5754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:15.633790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.5.122.62:40435)
0807 03:02:15.634792 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:15.634805 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:15.634851 (+ 46us) server_negotiation.cc:407] Connection header received
0807 03:02:15.645872 (+ 11021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:15.645892 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:15.646479 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:15.646820 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:15.651289 (+ 4469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:15.653234 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:15.656049 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:15.656810 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:15.657018 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:15.663978 (+ 6960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:15.664044 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:15.664083 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:15.664118 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:15.667314 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:15.668105 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:15.668112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:15.668116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:15.668277 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:15.668734 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:15.668737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:15.668739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:15.669466 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:15.669888 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:15.670739 (+ 851us) server_negotiation.cc:299] Negotiation successful
0807 03:02:15.671124 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":391,"threads_started":1}
I20260807 03:02:15.673836 5751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40435 } }
I20260807 03:02:15.674238 5751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:15.674563 5751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:15.674808 5751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:15.677408 5751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40435 } }
I20260807 03:02:15.678516 5751 leader_election.cc:302] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c5730ac9b642dba0019d98b2fdfe2a; no voters:
I20260807 03:02:15.679316 5751 leader_election.cc:288] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:15.679554 5758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:15.680303 5758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [term 1 LEADER]: Becoming Leader. State: Replica: 90c5730ac9b642dba0019d98b2fdfe2a, State: Running, Role: LEADER
I20260807 03:02:15.680959 5758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40435 } }
I20260807 03:02:15.682093 5751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:15.686399 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c5730ac9b642dba0019d98b2fdfe2a. Latest consensus state: current_term: 1 leader_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40435 } } }
I20260807 03:02:15.686182 5761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5730ac9b642dba0019d98b2fdfe2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40435 } } }
I20260807 03:02:15.687079 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:15.687134 5761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:15.690420 5774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:15.698891 5774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:15.714430 5774 catalog_manager.cc:1347] Generated new cluster ID: 18a9f5cff08344a7be1857e4d3cf4dea
I20260807 03:02:15.714555 5774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:15.725150 5774 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:15.726823 5774 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:15.736177 5774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c5730ac9b642dba0019d98b2fdfe2a: Generated new TSK 0
I20260807 03:02:15.736961 5774 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:02:15.791713 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40435
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:16.112794 5794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:16.113070 5794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:16.113175 5794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:16.120369 5794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:16.120476 5794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:16.120590 5794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:16.130395 5794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40435
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:16.131603 5794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:16.132867 5794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:16.146456 5801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:16.146451 5802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:16.146477 5805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:16.147225 5794 server_base.cc:1034] running on GCE node
I20260807 03:02:16.147804 5794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:16.148675 5794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:16.149873 5794 hybrid_clock.cc:648] HybridClock initialized: now 1786071736149830 us; error 29 us; skew 500 ppm
I20260807 03:02:16.150170 5794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:16 dist-test-slave-fgb3 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071736, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:16.153123 5794 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:16.154543 5794 webserver.cc:460] Webserver started at http://127.5.122.1:44139/ using document root <none> and password file <none>
I20260807 03:02:16.155172 5794 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:16.155289 5794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:16.155588 5794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:16.158437 5794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "645c57b75f2b4beb95de613bf974fe0f"
format_stamp: "Formatted at 2026-08-07 03:02:16 on dist-test-slave-fgb3"
server_key: "de1865d83984ed7334be5593cf3bc674"
server_key_iv: "9b40458cb9975370165b98529f88d2c5"
server_key_version: "encryptionkey@0"
I20260807 03:02:16.159166 5794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "645c57b75f2b4beb95de613bf974fe0f"
format_stamp: "Formatted at 2026-08-07 03:02:16 on dist-test-slave-fgb3"
server_key: "de1865d83984ed7334be5593cf3bc674"
server_key_iv: "9b40458cb9975370165b98529f88d2c5"
server_key_version: "encryptionkey@0"
I20260807 03:02:16.164005 5794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:02:16.167182 5814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:16.168352 5794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:02:16.168507 5794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "645c57b75f2b4beb95de613bf974fe0f"
format_stamp: "Formatted at 2026-08-07 03:02:16 on dist-test-slave-fgb3"
server_key: "de1865d83984ed7334be5593cf3bc674"
server_key_iv: "9b40458cb9975370165b98529f88d2c5"
server_key_version: "encryptionkey@0"
I20260807 03:02:16.168676 5794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:16.192113 5794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:16.196154 5794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:16.196416 5794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:16.197541 5794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:16.199198 5794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:16.199280 5794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:16.199469 5794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:16.199534 5794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:16.231475 5794 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:33593
I20260807 03:02:16.231534 5972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:33593 every 8 connection(s)
I20260807 03:02:16.232784 5794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:16.237654 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5794
I20260807 03:02:16.237898 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:16.238572 5608 external_mini_cluster.cc:1428] Setting key f4324ff213aec7591e947fb9e511ec5e
Aug 07 03:02:16 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071736, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:16.254673 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:16.236207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:33251 (local address 127.5.122.62:40435)
0807 03:02:16.236634 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:16.236638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:16.237601 (+ 963us) server_negotiation.cc:407] Connection header received
0807 03:02:16.239691 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:16.239697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:16.239792 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:16.239889 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:16.241458 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.242169 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.244055 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.244612 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.244740 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:16.248431 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:16.248438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:16.248442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:16.248482 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:16.251567 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:16.252294 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:16.252299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:16.252303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:16.252440 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:16.252837 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:16.252842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:16.252844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:16.253198 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:16.253351 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:16.254034 (+ 683us) server_negotiation.cc:299] Negotiation successful
0807 03:02:16.254265 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":224,"threads_started":1}
I20260807 03:02:16.255474 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:16.236727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40435 (local address 127.5.122.1:33251)
0807 03:02:16.237454 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:16.237491 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:16.239445 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:16.240122 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:16.240131 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:16.240737 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:16.241299 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:16.241317 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.242357 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.242363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:16.243900 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:16.243915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.244816 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.244822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:16.245062 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:16.245741 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:16.245773 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:16.248200 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:16.251714 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:16.251723 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:16.251736 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:16.252163 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:16.252531 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:16.252534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:16.252536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:16.252718 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:16.253325 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:16.253332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:16.253656 (+ 324us) client_negotiation.cc:769] Sending connection context
0807 03:02:16.253881 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 03:02:16.254279 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":271,"threads_started":1}
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
I20260807 03:02:16.257123 5974 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40435
I20260807 03:02:16.257581 5974 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:16.258518 5974 heartbeater.cc:511] Master 127.5.122.62:40435 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
I20260807 03:02:16.262449 5688 ts_manager.cc:194] Registered new tserver with Master: 645c57b75f2b4beb95de613bf974fe0f (127.5.122.1:33593)
I20260807 03:02:16.264608 5688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:33251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:02:16.281327 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40435
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:16.600495 5980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:16.600757 5980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:16.600895 5980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:16.608165 5980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:16.608283 5980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:16.608455 5980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:16.618093 5980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40435
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:16.619236 5980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:16.620489 5980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:16.632942 5989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:16.632983 5991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:16.632877 5993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:16.633776 5980 server_base.cc:1034] running on GCE node
I20260807 03:02:16.634407 5980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:16.635334 5980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:16.636524 5980 hybrid_clock.cc:648] HybridClock initialized: now 1786071736636482 us; error 41 us; skew 500 ppm
I20260807 03:02:16.636832 5980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:16 dist-test-slave-fgb3 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071736, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:16.639977 5980 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:16.641345 5980 webserver.cc:460] Webserver started at http://127.5.122.2:33285/ using document root <none> and password file <none>
I20260807 03:02:16.641984 5980 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:16.642086 5980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:16.642390 5980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:16.645202 5980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "fb25871efd874a81a0db23c25ad36833"
format_stamp: "Formatted at 2026-08-07 03:02:16 on dist-test-slave-fgb3"
server_key: "c5e122035c7bacbd4545d11fd100abac"
server_key_iv: "a90b160b73fdfc33e2bb3ff9033414d2"
server_key_version: "encryptionkey@0"
I20260807 03:02:16.645915 5980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "fb25871efd874a81a0db23c25ad36833"
format_stamp: "Formatted at 2026-08-07 03:02:16 on dist-test-slave-fgb3"
server_key: "c5e122035c7bacbd4545d11fd100abac"
server_key_iv: "a90b160b73fdfc33e2bb3ff9033414d2"
server_key_version: "encryptionkey@0"
I20260807 03:02:16.651021 5980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 03:02:16.654448 6006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:16.655692 5980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:02:16.655833 5980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "fb25871efd874a81a0db23c25ad36833"
format_stamp: "Formatted at 2026-08-07 03:02:16 on dist-test-slave-fgb3"
server_key: "c5e122035c7bacbd4545d11fd100abac"
server_key_iv: "a90b160b73fdfc33e2bb3ff9033414d2"
server_key_version: "encryptionkey@0"
I20260807 03:02:16.656013 5980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:16.683977 5980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:16.687908 5980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:16.688174 5980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:16.689121 5980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:16.690534 5980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:16.690623 5980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:16.690745 5980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:16.690795 5980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:16.722821 5980 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:41817
I20260807 03:02:16.722880 6151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:41817 every 8 connection(s)
I20260807 03:02:16.724092 5980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:16.728281 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5980
I20260807 03:02:16.728467 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:16.728884 5608 external_mini_cluster.cc:1428] Setting key efcb0829765186976f6ffb35fb2a8186
Aug 07 03:02:16 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071736, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:16.744218 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:16.726716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:37095 (local address 127.5.122.62:40435)
0807 03:02:16.726930 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:16.726937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:16.727993 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 03:02:16.729910 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:16.729916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:16.729999 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:16.730090 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:16.731634 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.732350 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.733861 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.734276 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.734436 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:16.738186 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:16.738191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:16.738195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:16.738226 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:16.740695 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:16.741808 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:16.741812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:16.741815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:16.741944 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:16.742438 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:16.742441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:16.742443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:16.742720 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:16.742835 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:16.743785 (+ 950us) server_negotiation.cc:299] Negotiation successful
0807 03:02:16.743971 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 03:02:16.745678 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:16.727118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40435 (local address 127.5.122.2:37095)
0807 03:02:16.727809 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:16.727879 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:16.729629 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:16.730338 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:16.730350 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:16.730927 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:16.731476 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:16.731494 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.732489 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.732493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:16.733695 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:16.733712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:16.734522 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:16.734529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:16.734785 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:16.735430 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:16.735452 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:16.737928 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:16.740843 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:16.740852 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:16.740868 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:16.741436 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:16.742058 (+ 622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:16.742063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:16.742066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:16.742342 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:16.742869 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:16.742877 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:16.743320 (+ 443us) client_negotiation.cc:769] Sending connection context
0807 03:02:16.743641 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 03:02:16.744218 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":290,"threads_started":1}
I20260807 03:02:16.747174 6152 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40435
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
I20260807 03:02:16.747512 6152 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:16.748283 6152 heartbeater.cc:511] Master 127.5.122.62:40435 requested a full tablet report, sending...
I20260807 03:02:16.749745 5688 ts_manager.cc:194] Registered new tserver with Master: fb25871efd874a81a0db23c25ad36833 (127.5.122.2:41817)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
I20260807 03:02:16.750710 5688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:37095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:16.771430 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40435
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:17.086978 6157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:17.087240 6157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:17.087386 6157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:17.094535 6157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:17.094651 6157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:17.094801 6157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:17.104228 6157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40435
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:17.105427 6157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:17.106786 6157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:17.118884 6166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:17.118847 6168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:17.118916 6171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:17.120054 6157 server_base.cc:1034] running on GCE node
I20260807 03:02:17.120610 6157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:17.121464 6157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:17.122654 6157 hybrid_clock.cc:648] HybridClock initialized: now 1786071737122602 us; error 49 us; skew 500 ppm
I20260807 03:02:17.122946 6157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:17.125880 6157 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:17.127234 6157 webserver.cc:460] Webserver started at http://127.5.122.3:45017/ using document root <none> and password file <none>
I20260807 03:02:17.127861 6157 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:17.127981 6157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:17.128268 6157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:17.131048 6157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "bff9cfe378134dba95f17aede185aacf"
format_stamp: "Formatted at 2026-08-07 03:02:17 on dist-test-slave-fgb3"
server_key: "e0b7c7d3f4f64be42fa5451468b30801"
server_key_iv: "2dfd2f27360d8468905d1e7dfe97b033"
server_key_version: "encryptionkey@0"
I20260807 03:02:17.131770 6157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "bff9cfe378134dba95f17aede185aacf"
format_stamp: "Formatted at 2026-08-07 03:02:17 on dist-test-slave-fgb3"
server_key: "e0b7c7d3f4f64be42fa5451468b30801"
server_key_iv: "2dfd2f27360d8468905d1e7dfe97b033"
server_key_version: "encryptionkey@0"
I20260807 03:02:17.136657 6157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:02:17.139889 6182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:17.141076 6157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 03:02:17.141220 6157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "bff9cfe378134dba95f17aede185aacf"
format_stamp: "Formatted at 2026-08-07 03:02:17 on dist-test-slave-fgb3"
server_key: "e0b7c7d3f4f64be42fa5451468b30801"
server_key_iv: "2dfd2f27360d8468905d1e7dfe97b033"
server_key_version: "encryptionkey@0"
I20260807 03:02:17.141392 6157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:17.157658 6157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:17.161571 6157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:17.161818 6157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:17.162722 6157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:17.164052 6157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:17.164129 6157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:17.164245 6157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:17.164299 6157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:17.194794 6157 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:38011
I20260807 03:02:17.194836 6321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:38011 every 8 connection(s)
I20260807 03:02:17.195979 6157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:17.197611 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6157
I20260807 03:02:17.197788 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:17.198355 5608 external_mini_cluster.cc:1428] Setting key ca9dedf9dedc61ce058f6f3e4299222b
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:17.215184 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:17.198636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:42763 (local address 127.5.122.62:40435)
0807 03:02:17.198797 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:17.198801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:17.200334 (+ 1533us) server_negotiation.cc:407] Connection header received
0807 03:02:17.202127 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:17.202133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:17.202217 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:17.202302 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:17.203753 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.204533 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.206040 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.206516 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.206626 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:17.209886 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:17.209891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:17.209894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:17.209929 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:17.212310 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.212965 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.212971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.212975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.213102 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.213504 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.213510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.213513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.213795 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:17.213927 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:17.214609 (+ 682us) server_negotiation.cc:299] Negotiation successful
0807 03:02:17.214845 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 03:02:17.216133 6325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:17.199335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40435 (local address 127.5.122.3:42763)
0807 03:02:17.200139 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:17.200207 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:17.201876 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:17.202557 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:17.202565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:17.203056 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:17.203593 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:17.203611 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.204677 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.204681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:17.205904 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:17.205916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.206737 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.206745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:17.206976 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:17.207578 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:17.207604 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:17.209685 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:17.212438 (+ 2753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:17.212447 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:17.212462 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:17.212856 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:17.213203 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:17.213208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:17.213212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:17.213397 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:17.213917 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:17.213925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:17.214229 (+ 304us) client_negotiation.cc:769] Sending connection context
0807 03:02:17.214524 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 03:02:17.215026 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":344,"threads_started":1}
I20260807 03:02:17.217448 6323 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40435
I20260807 03:02:17.217728 6323 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:17.218508 6323 heartbeater.cc:511] Master 127.5.122.62:40435 requested a full tablet report, sending...
I20260807 03:02:17.219931 5688 ts_manager.cc:194] Registered new tserver with Master: bff9cfe378134dba95f17aede185aacf (127.5.122.3:38011)
I20260807 03:02:17.220681 5688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:42763
I20260807 03:02:17.226684 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:17.268568 5974 heartbeater.cc:503] Master 127.5.122.62:40435 was elected leader, sending a full tablet report...
I20260807 03:02:17.272454 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:17.257162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.5.122.1:33593)
0807 03:02:17.257858 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:17.257869 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:17.257901 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:02:17.257970 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:17.257977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:17.258207 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:17.258388 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:17.260010 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.261125 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.262479 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.263034 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.263136 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:17.266926 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:17.266937 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:17.266949 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:17.266986 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:17.269883 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.270616 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.270623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.270625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.270747 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.271049 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.271052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.271053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.271470 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:17.271711 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:17.271894 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 03:02:17.272090 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":336,"threads_started":1}
W20260807 03:02:17.273756 5856 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50318
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:17.317543 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:17.303796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.5.122.1:33593)
0807 03:02:17.304030 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:17.304036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:17.304261 (+ 225us) server_negotiation.cc:407] Connection header received
0807 03:02:17.304406 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:17.304410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:17.304512 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:17.304598 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:17.305860 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.306622 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.307742 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.308139 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.308224 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:17.311819 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:17.311824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:17.311827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:17.311861 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:17.314599 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.315264 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.315270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.315274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.315433 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.315838 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.315851 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.315854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.316206 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:17.316442 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:17.316960 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 03:02:17.317206 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:02:17.319852 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5794
I20260807 03:02:17.332002 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:17.320999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.5.122.1:33593)
0807 03:02:17.321195 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:17.321201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:17.321437 (+ 236us) server_negotiation.cc:407] Connection header received
0807 03:02:17.321532 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:17.321535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:17.321587 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:17.321657 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:17.322798 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.323511 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.324653 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.325109 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.325217 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:17.327244 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:17.327249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:17.327252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:17.327284 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:17.329826 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.330419 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.330422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.330424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.330556 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.330833 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.330836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.330838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.331080 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:17.331274 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:17.331479 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 03:02:17.331632 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:02:17.332763 5965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:17.419476 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5794
I20260807 03:02:17.439311 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5980
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:02:17.454527 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:17.440221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33728 (local address 127.5.122.2:41817)
0807 03:02:17.440766 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:17.440774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:17.440803 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:02:17.440873 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:17.440880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:17.441103 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:17.441244 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:17.442393 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.443428 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.444657 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.445266 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.445377 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:17.449217 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:17.449228 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:17.449242 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:17.449286 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:17.452050 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.452615 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.452621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.452623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.452760 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.453138 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.453142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.453144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.453480 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:17.453699 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:17.453944 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 03:02:17.454146 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":259,"threads_started":1}
I20260807 03:02:17.455775 6146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:17.536616 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5980
I20260807 03:02:17.555188 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6157
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:02:17.570894 6343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:17.556281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.5.122.3:38011)
0807 03:02:17.556985 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:17.556992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:17.557021 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:02:17.557085 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:17.557092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:17.557311 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:17.557445 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:17.558735 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.559786 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.561065 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:17.561697 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:17.561829 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:17.565761 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:17.565773 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:17.565787 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:17.565824 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:17.568533 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.569116 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.569122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.569124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.569242 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:17.569506 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:17.569509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:17.569511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:17.569842 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:17.570067 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:17.570271 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 03:02:17.570550 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":369,"threads_started":1}
I20260807 03:02:17.572196 6317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:17.655489 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6157
I20260807 03:02:17.676620 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5636
I20260807 03:02:17.677816 5744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:17.780180 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5636
2026-08-07T03:02:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4002 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[6350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[6350](info): set up 2 sockets
Aug 07 03:02:17 dist-test-slave-fgb3 krb5kdc[6350](info): commencing operation
krb5kdc: starting...
W20260807 03:02:18.949054 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.004s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:19 dist-test-slave-fgb3 krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:19.126896 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45593
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:45593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:19.436885 6365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:19.437142 6365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:19.437227 6365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:19.444177 6365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:19.444281 6365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:19.444326 6365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:19.444378 6365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:19.444420 6365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:19.453595 6365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:45593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:19.454726 6365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:19.455932 6365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:19.467020 6376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:19.466991 6378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:19.467272 6380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:19.468014 6365 server_base.cc:1034] running on GCE node
I20260807 03:02:19.468779 6365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:19.470180 6365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:19.471429 6365 hybrid_clock.cc:648] HybridClock initialized: now 1786071739471366 us; error 56 us; skew 500 ppm
I20260807 03:02:19.471813 6365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:19 dist-test-slave-fgb3 krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071739, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:19.477465 6365 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:19.478861 6365 webserver.cc:460] Webserver started at http://127.5.122.62:37539/ using document root <none> and password file <none>
I20260807 03:02:19.479506 6365 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:19.479619 6365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:19.479905 6365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:19.482834 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "8b69bba48e3a446794e093f45b98e627"
format_stamp: "Formatted at 2026-08-07 03:02:19 on dist-test-slave-fgb3"
server_key: "bb875239753eda6a288f5f6ad4de2705"
server_key_iv: "c0787741cf47162c70ff709e887392bc"
server_key_version: "encryptionkey@0"
I20260807 03:02:19.483546 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "8b69bba48e3a446794e093f45b98e627"
format_stamp: "Formatted at 2026-08-07 03:02:19 on dist-test-slave-fgb3"
server_key: "bb875239753eda6a288f5f6ad4de2705"
server_key_iv: "c0787741cf47162c70ff709e887392bc"
server_key_version: "encryptionkey@0"
I20260807 03:02:19.488340 6365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 03:02:19.491534 6389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:19.492758 6365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:02:19.492915 6365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "8b69bba48e3a446794e093f45b98e627"
format_stamp: "Formatted at 2026-08-07 03:02:19 on dist-test-slave-fgb3"
server_key: "bb875239753eda6a288f5f6ad4de2705"
server_key_iv: "c0787741cf47162c70ff709e887392bc"
server_key_version: "encryptionkey@0"
I20260807 03:02:19.493094 6365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:19.505481 6365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:19.509482 6365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:19.509730 6365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:19.526551 6365 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:45593
I20260807 03:02:19.526575 6465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:45593 every 8 connection(s)
I20260807 03:02:19.527889 6365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:19.531572 6467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:19.532678 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6365
I20260807 03:02:19.532914 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:19.533777 5608 external_mini_cluster.cc:1428] Setting key 91ad78135f14f04002a57540fef40d2f
I20260807 03:02:19.545723 6467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627: Bootstrap starting.
Aug 07 03:02:19 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:19.549338 6467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:19.550382 6467 log.cc:824] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:19.553457 6467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627: No bootstrap required, opened a new log
I20260807 03:02:19.556241 6472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:19.535873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.5.122.62:45593)
0807 03:02:19.536657 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:19.536671 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:19.536718 (+ 47us) server_negotiation.cc:407] Connection header received
0807 03:02:19.538298 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:19.538343 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:19.538724 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:19.539044 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:19.540695 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:19.541933 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:19.543681 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:19.544313 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:19.544437 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:19.549079 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:19.549087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:19.549100 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:19.549134 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:19.552359 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:19.553099 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:19.553104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:19.553109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:19.553273 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:19.553647 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:19.553651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:19.553653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:19.554164 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:19.554434 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:19.554920 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 03:02:19.555242 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":395,"threads_started":1}
I20260807 03:02:19.561102 6467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b69bba48e3a446794e093f45b98e627" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45593 } }
I20260807 03:02:19.561436 6467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:19.561548 6467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b69bba48e3a446794e093f45b98e627, State: Initialized, Role: FOLLOWER
I20260807 03:02:19.562105 6467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b69bba48e3a446794e093f45b98e627" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45593 } }
I20260807 03:02:19.562333 6467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:19.562453 6467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:19.562570 6467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:19.564559 6467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b69bba48e3a446794e093f45b98e627" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45593 } }
I20260807 03:02:19.564963 6467 leader_election.cc:302] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b69bba48e3a446794e093f45b98e627; no voters:
I20260807 03:02:19.565449 6467 leader_election.cc:288] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:19.565575 6474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:19.566183 6474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [term 1 LEADER]: Becoming Leader. State: Replica: 8b69bba48e3a446794e093f45b98e627, State: Running, Role: LEADER
I20260807 03:02:19.566838 6474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b69bba48e3a446794e093f45b98e627" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45593 } }
I20260807 03:02:19.567521 6467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:19.569545 6477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b69bba48e3a446794e093f45b98e627. Latest consensus state: current_term: 1 leader_uuid: "8b69bba48e3a446794e093f45b98e627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b69bba48e3a446794e093f45b98e627" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45593 } } }
I20260807 03:02:19.569518 6475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b69bba48e3a446794e093f45b98e627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b69bba48e3a446794e093f45b98e627" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45593 } } }
I20260807 03:02:19.569937 6475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:19.569937 6477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:19.575770 6482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:19.579882 6482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:19.588053 6482 catalog_manager.cc:1347] Generated new cluster ID: 55dab4d88d8c4bc9b24baeb1c1f0cf90
I20260807 03:02:19.588150 6482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:19.604749 6482 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:19.605989 6482 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:19.612740 6482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b69bba48e3a446794e093f45b98e627: Generated new TSK 0
I20260807 03:02:19.613482 6482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:02:19.657936 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45593
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:19.970764 6507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:19.971042 6507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:19.971185 6507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:19.978134 6507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:19.978243 6507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:19.978408 6507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:19.988701 6507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:19.989869 6507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:19.991256 6507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:20.003410 6518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:20.003590 6516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:20.003707 6520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:20.004638 6507 server_base.cc:1034] running on GCE node
I20260807 03:02:20.005203 6507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:20.006103 6507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:20.007292 6507 hybrid_clock.cc:648] HybridClock initialized: now 1786071740007214 us; error 73 us; skew 500 ppm
I20260807 03:02:20.007586 6507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:20 dist-test-slave-fgb3 krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:20.010581 6507 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:20.011898 6507 webserver.cc:460] Webserver started at http://127.5.122.1:33691/ using document root <none> and password file <none>
I20260807 03:02:20.012524 6507 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:20.012619 6507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:20.012887 6507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:20.015740 6507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "6079fbfc3ef84e149e87316f28948c85"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "bc61f4d49f5632829e2fc1af3177bf1c"
server_key_iv: "d01e6d463a166a3990aea7212cd42c5c"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.016491 6507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "6079fbfc3ef84e149e87316f28948c85"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "bc61f4d49f5632829e2fc1af3177bf1c"
server_key_iv: "d01e6d463a166a3990aea7212cd42c5c"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.021288 6507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 03:02:20.024637 6532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.025827 6507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 03:02:20.025978 6507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "6079fbfc3ef84e149e87316f28948c85"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "bc61f4d49f5632829e2fc1af3177bf1c"
server_key_iv: "d01e6d463a166a3990aea7212cd42c5c"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.026149 6507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:20.039350 6507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:20.042977 6507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:20.043211 6507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:20.044073 6507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:20.045441 6507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:20.045532 6507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.045684 6507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:20.045750 6507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.077561 6507 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:44101
I20260807 03:02:20.077610 6693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:44101 every 8 connection(s)
I20260807 03:02:20.078893 6507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:20.084869 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6507
I20260807 03:02:20.085081 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:20.085685 5608 external_mini_cluster.cc:1428] Setting key 964bdefeb57c18a8b405eb851b5d9536
Aug 07 03:02:20 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:20.099472 6698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:20.081477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:51253 (local address 127.5.122.62:45593)
0807 03:02:20.081884 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:20.081889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:20.082909 (+ 1020us) server_negotiation.cc:407] Connection header received
0807 03:02:20.084717 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:20.084723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:20.084805 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:20.084908 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:20.086589 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.087409 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.089566 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.089968 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.090060 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:20.093771 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:20.093777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:20.093781 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:20.093822 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:20.096257 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:20.097112 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:20.097117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:20.097119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:20.097240 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:20.097730 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:20.097735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:20.097737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:20.098101 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:20.098239 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:20.098927 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 03:02:20.099128 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":213,"threads_started":1}
I20260807 03:02:20.100255 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:20.082009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45593 (local address 127.5.122.1:51253)
0807 03:02:20.082753 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:20.082797 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:20.084459 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:20.085148 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:20.085160 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:20.085728 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:20.086418 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:20.086445 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.087562 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.087567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:20.089424 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:20.089439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.090179 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.090187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:20.090480 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:20.091159 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:20.091189 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:20.093557 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:20.096399 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:20.096408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:20.096422 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:20.097003 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:20.097354 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:20.097359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:20.097363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:20.097597 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:20.098206 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:20.098212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:20.098571 (+ 359us) client_negotiation.cc:769] Sending connection context
0807 03:02:20.098788 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 03:02:20.099149 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":320,"threads_started":1}
I20260807 03:02:20.101806 6695 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45593
I20260807 03:02:20.102104 6695 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:20.102813 6695 heartbeater.cc:511] Master 127.5.122.62:45593 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
I20260807 03:02:20.105041 6413 ts_manager.cc:194] Registered new tserver with Master: 6079fbfc3ef84e149e87316f28948c85 (127.5.122.1:44101)
I20260807 03:02:20.106504 6413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:51253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:02:20.126675 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45593
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:20.446338 6702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:20.446604 6702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:20.446733 6702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:20.453585 6702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:20.453676 6702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:20.453821 6702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:20.462950 6702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:20.464071 6702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:20.465315 6702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:20.477713 6713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:20.477756 6717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:20.477780 6714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:20.478518 6702 server_base.cc:1034] running on GCE node
I20260807 03:02:20.479061 6702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:20.479938 6702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:20.481180 6702 hybrid_clock.cc:648] HybridClock initialized: now 1786071740481137 us; error 32 us; skew 500 ppm
I20260807 03:02:20.481472 6702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:20 dist-test-slave-fgb3 krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:20.484467 6702 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:20.485853 6702 webserver.cc:460] Webserver started at http://127.5.122.2:38047/ using document root <none> and password file <none>
I20260807 03:02:20.486578 6702 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:20.486698 6702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:20.487004 6702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:20.489888 6702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "a630f9b3c4814607ac398df889f6a0b2"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "9518f63483304ff38e938f4312421a95"
server_key_iv: "608055bbd307772e68f41d3aa47f1818"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.490722 6702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "a630f9b3c4814607ac398df889f6a0b2"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "9518f63483304ff38e938f4312421a95"
server_key_iv: "608055bbd307772e68f41d3aa47f1818"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.495805 6702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 03:02:20.499366 6727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.500640 6702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:02:20.500790 6702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "a630f9b3c4814607ac398df889f6a0b2"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "9518f63483304ff38e938f4312421a95"
server_key_iv: "608055bbd307772e68f41d3aa47f1818"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.500986 6702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:20.520447 6702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:20.524153 6702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:20.524406 6702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:20.525338 6702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:20.526737 6702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:20.526816 6702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.526943 6702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:20.526997 6702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.557940 6702 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:40925
I20260807 03:02:20.557983 6879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:40925 every 8 connection(s)
I20260807 03:02:20.559267 6702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:20.563505 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6702
I20260807 03:02:20.563684 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:20.564092 5608 external_mini_cluster.cc:1428] Setting key bf32dc1ea91a65d9a4b9a569386830bf
Aug 07 03:02:20 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:20.578876 6698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:20.561679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:35107 (local address 127.5.122.62:45593)
0807 03:02:20.561839 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:20.561854 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:20.563034 (+ 1180us) server_negotiation.cc:407] Connection header received
0807 03:02:20.564870 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:20.564876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:20.564960 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:20.565063 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:20.566892 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.567921 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.569504 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.570047 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.570164 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:20.573514 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:20.573519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:20.573522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:20.573555 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:20.575963 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:20.576642 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:20.576646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:20.576648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:20.576777 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:20.577191 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:20.577195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:20.577197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:20.577467 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:20.577600 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:20.578289 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 03:02:20.578556 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:02:20.579810 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:20.562089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45593 (local address 127.5.122.2:35107)
0807 03:02:20.562811 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:20.562917 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:20.564563 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:20.565334 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:20.565344 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:20.566081 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:20.566711 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:20.566729 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.568070 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.568075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:20.569369 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:20.569381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:20.570228 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:20.570233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:20.570463 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:20.571067 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:20.571094 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:20.573312 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:20.576096 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:20.576105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:20.576121 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:20.576518 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:20.576889 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:20.576894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:20.576897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:20.577077 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:20.577591 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:20.577598 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:20.577915 (+ 317us) client_negotiation.cc:769] Sending connection context
0807 03:02:20.578166 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 03:02:20.578710 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":309,"threads_started":1}
I20260807 03:02:20.581182 6880 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45593
I20260807 03:02:20.581475 6880 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
I20260807 03:02:20.582173 6880 heartbeater.cc:511] Master 127.5.122.62:45593 requested a full tablet report, sending...
I20260807 03:02:20.583652 6413 ts_manager.cc:194] Registered new tserver with Master: a630f9b3c4814607ac398df889f6a0b2 (127.5.122.2:40925)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
I20260807 03:02:20.584682 6413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:35107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:20.605441 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45593
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:20.916776 6891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:20.917018 6891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:20.917155 6891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:20.924166 6891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:20.924271 6891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:20.924431 6891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:20.933626 6891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:20.934762 6891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:20.936002 6891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:20.947778 6902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:20.947911 6905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:20.947817 6903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:20.948686 6891 server_base.cc:1034] running on GCE node
I20260807 03:02:20.949262 6891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:20.950081 6891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:20.951268 6891 hybrid_clock.cc:648] HybridClock initialized: now 1786071740951220 us; error 36 us; skew 500 ppm
I20260807 03:02:20.951548 6891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:20 dist-test-slave-fgb3 krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:20.954589 6891 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:20.955876 6891 webserver.cc:460] Webserver started at http://127.5.122.3:42573/ using document root <none> and password file <none>
I20260807 03:02:20.956445 6891 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:20.956539 6891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:20.956777 6891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:20.959579 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "54ac1b9a849d49f199898b356fbada98"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "f5f666d51b2f4458a6b6a18bdb4cc070"
server_key_iv: "dc1cf954bb1bfab9a2ac6f8a8ff6825c"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.960290 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "54ac1b9a849d49f199898b356fbada98"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "f5f666d51b2f4458a6b6a18bdb4cc070"
server_key_iv: "dc1cf954bb1bfab9a2ac6f8a8ff6825c"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.964991 6891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:02:20.968186 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.969435 6891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:20.969570 6891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "54ac1b9a849d49f199898b356fbada98"
format_stamp: "Formatted at 2026-08-07 03:02:20 on dist-test-slave-fgb3"
server_key: "f5f666d51b2f4458a6b6a18bdb4cc070"
server_key_iv: "dc1cf954bb1bfab9a2ac6f8a8ff6825c"
server_key_version: "encryptionkey@0"
I20260807 03:02:20.969731 6891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:20.988303 6891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:20.992115 6891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:20.992370 6891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:20.993271 6891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:20.994673 6891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:20.994758 6891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:20.994870 6891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:20.994920 6891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:21.024559 6891 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36205
I20260807 03:02:21.024595 7073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36205 every 8 connection(s)
I20260807 03:02:21.025779 6891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:21.030489 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6891
I20260807 03:02:21.030684 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:21.031102 5608 external_mini_cluster.cc:1428] Setting key dfdc4cff31056e728c9c8ba1f166ea5a
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:21.044173 6698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.028263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:54361 (local address 127.5.122.62:45593)
0807 03:02:21.028476 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.028482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.029622 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 03:02:21.031690 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.031697 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.031785 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.031875 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.033466 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.034361 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.035788 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.036189 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.036274 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.039356 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.039361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.039364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.039396 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.041818 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.042443 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.042446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.042449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.042565 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.042889 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.042892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.042894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.043131 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.043235 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.043773 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.043933 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 03:02:21.045188 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.028754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45593 (local address 127.5.122.3:54361)
0807 03:02:21.029480 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:21.029524 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:21.031441 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:21.032124 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:21.032135 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:21.032672 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:21.033312 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.033330 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.034492 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.034496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.035684 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.035696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.036353 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.036358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.036536 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.037080 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:21.037105 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:21.039175 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:21.041920 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:21.041925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:21.041935 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:21.042361 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:21.042640 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:21.042643 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:21.042645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:21.042811 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:21.043214 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:21.043219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:21.043505 (+ 286us) client_negotiation.cc:769] Sending connection context
0807 03:02:21.043713 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 03:02:21.044073 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":315,"threads_started":1}
I20260807 03:02:21.046402 7074 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45593
I20260807 03:02:21.046662 7074 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:21.047312 7074 heartbeater.cc:511] Master 127.5.122.62:45593 requested a full tablet report, sending...
I20260807 03:02:21.048557 6413 ts_manager.cc:194] Registered new tserver with Master: 54ac1b9a849d49f199898b356fbada98 (127.5.122.3:36205)
I20260807 03:02:21.049301 6413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:54361
I20260807 03:02:21.058054 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:02:21.077656 6698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.065183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55780 (local address 127.5.122.62:45593)
0807 03:02:21.065341 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.065345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.065866 (+ 521us) server_negotiation.cc:407] Connection header received
0807 03:02:21.066037 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.066041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.066104 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.066187 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.067703 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.068501 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.069749 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.070167 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.070291 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.072690 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.072695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.072698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.072738 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.075181 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.075781 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.075786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.075788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.075918 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.076252 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.076258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.076260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.076568 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.076794 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.077141 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.077363 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 03:02:21.086427 6413 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:21.094352 6413 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:21.109252 6695 heartbeater.cc:503] Master 127.5.122.62:45593 was elected leader, sending a full tablet report...
I20260807 03:02:21.134832 7097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.110143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36205 (local address 127.0.0.1:33204)
0807 03:02:21.124728 (+ 14585us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:21.124803 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:21.125121 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:21.126501 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:21.126508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:21.126538 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:21.127291 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.127307 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.129540 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.129545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.131894 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.131907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.133726 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.133733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.133942 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.133996 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 03:02:21.134114 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 03:02:21.134348 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14323,"mutex_wait_us":5,"thread_start_us":15231,"threads_started":1}
I20260807 03:02:21.135293 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.110469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.5.122.3:36205)
0807 03:02:21.125696 (+ 15227us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.125705 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.125737 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:02:21.125827 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.125834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.126076 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.127495 (+ 1419us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:21.127780 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.129391 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.132420 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.133488 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.134530 (+ 1042us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.134752 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.134825 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.134983 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15064,"thread_start_us":15735,"threads_started":1}
I20260807 03:02:21.135038 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.109458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44101 (local address 127.0.0.1:49216)
0807 03:02:21.124728 (+ 15270us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:21.125310 (+ 582us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:21.125478 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:21.126530 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:21.126535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:21.126565 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:21.127201 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.127216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.129356 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.129362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.132200 (+ 2838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.132213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.134107 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.134113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.134334 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.134365 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 03:02:21.134441 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 03:02:21.134567 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14911,"mutex_wait_us":24,"thread_start_us":15709,"threads_started":1}
I20260807 03:02:21.135948 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.109901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.5.122.1:44101)
0807 03:02:21.125230 (+ 15329us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.125240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.125612 (+ 372us) server_negotiation.cc:407] Connection header received
0807 03:02:21.125727 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.125735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.126009 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.126217 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:21.127427 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.129173 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.132360 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.133597 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.133715 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.133963 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.135187 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.135355 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15172,"thread_start_us":15023,"threads_started":1}
I20260807 03:02:21.139492 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.126866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:40925 (local address 127.0.0.1:41924)
0807 03:02:21.131419 (+ 4553us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:21.131435 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:21.131547 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:21.132399 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:21.132403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:21.132427 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:21.132751 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.132761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.134609 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.134613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.136628 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.136643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.138784 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.138791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.138956 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.138987 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 03:02:21.139071 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 03:02:21.139185 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4417,"thread_start_us":4275,"threads_started":1}
I20260807 03:02:21.139900 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.126269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41924 (local address 127.5.122.2:40925)
0807 03:02:21.130971 (+ 4702us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.130979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.131665 (+ 686us) server_negotiation.cc:407] Connection header received
0807 03:02:21.131764 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.131771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.131987 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.132166 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:21.132939 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.134457 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.136808 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.138199 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.138347 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.138525 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.139457 (+ 932us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.139619 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4554,"thread_start_us":4416,"threads_started":1}
I20260807 03:02:21.139575 6981 tablet_service.cc:1467] Processing CreateTablet for tablet cb4179f48d374276b5c73e18811281eb (DEFAULT_TABLE table=test-table [id=b9f66cb0465f4ffbb7c9198645e2ad82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:21.139571 6608 tablet_service.cc:1467] Processing CreateTablet for tablet cb4179f48d374276b5c73e18811281eb (DEFAULT_TABLE table=test-table [id=b9f66cb0465f4ffbb7c9198645e2ad82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:21.141147 6608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4179f48d374276b5c73e18811281eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:21.141170 6981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4179f48d374276b5c73e18811281eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:21.143216 6795 tablet_service.cc:1467] Processing CreateTablet for tablet cb4179f48d374276b5c73e18811281eb (DEFAULT_TABLE table=test-table [id=b9f66cb0465f4ffbb7c9198645e2ad82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:21.144450 6795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4179f48d374276b5c73e18811281eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:21.153002 7103 tablet_bootstrap.cc:492] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85: Bootstrap starting.
I20260807 03:02:21.156159 7103 tablet_bootstrap.cc:654] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:21.157152 7103 log.cc:824] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:21.158078 7104 tablet_bootstrap.cc:492] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98: Bootstrap starting.
I20260807 03:02:21.162258 7104 tablet_bootstrap.cc:654] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:21.163125 7106 tablet_bootstrap.cc:492] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2: Bootstrap starting.
I20260807 03:02:21.163614 7104 log.cc:824] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:21.166133 7103 tablet_bootstrap.cc:492] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85: No bootstrap required, opened a new log
I20260807 03:02:21.166416 7103 ts_tablet_manager.cc:1397] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.011s
I20260807 03:02:21.166503 7106 tablet_bootstrap.cc:654] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:21.166635 7104 tablet_bootstrap.cc:492] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98: No bootstrap required, opened a new log
I20260807 03:02:21.166900 7104 ts_tablet_manager.cc:1397] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 03:02:21.167729 7106 log.cc:824] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:21.170954 7106 tablet_bootstrap.cc:492] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2: No bootstrap required, opened a new log
I20260807 03:02:21.171254 7106 ts_tablet_manager.cc:1397] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 03:02:21.174752 7103 raft_consensus.cc:348] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.175104 7103 raft_consensus.cc:374] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:21.175230 7103 raft_consensus.cc:729] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6079fbfc3ef84e149e87316f28948c85, State: Initialized, Role: FOLLOWER
I20260807 03:02:21.175814 7103 consensus_queue.cc:260] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.176741 7104 raft_consensus.cc:348] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.177198 7104 raft_consensus.cc:374] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:21.177327 7104 raft_consensus.cc:729] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ac1b9a849d49f199898b356fbada98, State: Initialized, Role: FOLLOWER
I20260807 03:02:21.177546 7103 ts_tablet_manager.cc:1428] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260807 03:02:21.178011 7104 consensus_queue.cc:260] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.179061 7074 heartbeater.cc:503] Master 127.5.122.62:45593 was elected leader, sending a full tablet report...
I20260807 03:02:21.179855 7104 ts_tablet_manager.cc:1428] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 03:02:21.181371 7109 raft_consensus.cc:482] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:21.181188 7106 raft_consensus.cc:348] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.181624 7106 raft_consensus.cc:374] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:21.181742 7106 raft_consensus.cc:729] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a630f9b3c4814607ac398df889f6a0b2, State: Initialized, Role: FOLLOWER
I20260807 03:02:21.181587 7109 raft_consensus.cc:504] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.182348 7106 consensus_queue.cc:260] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.184010 7106 ts_tablet_manager.cc:1428] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 03:02:21.185776 6880 heartbeater.cc:503] Master 127.5.122.62:45593 was elected leader, sending a full tablet report...
I20260807 03:02:21.188088 7109 leader_election.cc:288] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a630f9b3c4814607ac398df889f6a0b2 (127.5.122.2:40925), 54ac1b9a849d49f199898b356fbada98 (127.5.122.3:36205)
I20260807 03:02:21.193442 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.185492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:34821 (local address 127.5.122.3:36205)
0807 03:02:21.185671 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.185677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.187638 (+ 1961us) server_negotiation.cc:407] Connection header received
0807 03:02:21.187921 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.187926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.187994 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.188055 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:21.188869 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.189905 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.191816 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.192571 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.192679 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.192788 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.193084 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.193210 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 03:02:21.193604 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.185649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:45875 (local address 127.5.122.2:40925)
0807 03:02:21.187090 (+ 1441us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.187097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.188234 (+ 1137us) server_negotiation.cc:407] Connection header received
0807 03:02:21.188637 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.188642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.188714 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.188820 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:21.189337 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.190298 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.191985 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.192748 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.192875 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.192966 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.193250 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.193373 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1292}
I20260807 03:02:21.193672 7115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.185043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36205 (local address 127.5.122.1:34821)
0807 03:02:21.187542 (+ 2499us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:21.187561 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:21.187750 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:21.188258 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:21.188263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:21.188299 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:21.188715 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.188728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.190069 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.190074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.191663 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.191676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.192774 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.192781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.192939 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.192967 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 03:02:21.193047 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 03:02:21.193178 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2372,"mutex_wait_us":1,"spinlock_wait_cycles":9344,"thread_start_us":2803,"threads_started":1}
I20260807 03:02:21.193852 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.185075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:40925 (local address 127.5.122.1:45875)
0807 03:02:21.188137 (+ 3062us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:21.188153 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:21.188477 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:21.188840 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:21.188844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:21.188869 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:21.189201 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.189213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.190435 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.190439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.191833 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:21.191846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.192968 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.192974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:21.193106 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.193129 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:02:21.193208 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 03:02:21.193328 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2935,"spinlock_wait_cycles":896,"thread_start_us":2126,"threads_started":1}
I20260807 03:02:21.194756 6820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4179f48d374276b5c73e18811281eb" candidate_uuid: "6079fbfc3ef84e149e87316f28948c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a630f9b3c4814607ac398df889f6a0b2" is_pre_election: true
I20260807 03:02:21.194741 7005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4179f48d374276b5c73e18811281eb" candidate_uuid: "6079fbfc3ef84e149e87316f28948c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ac1b9a849d49f199898b356fbada98" is_pre_election: true
I20260807 03:02:21.195446 6820 raft_consensus.cc:2454] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6079fbfc3ef84e149e87316f28948c85 in term 0.
I20260807 03:02:21.196147 6533 leader_election.cc:302] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6079fbfc3ef84e149e87316f28948c85, a630f9b3c4814607ac398df889f6a0b2; no voters:
I20260807 03:02:21.196509 7109 raft_consensus.cc:2781] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:21.196630 7109 raft_consensus.cc:482] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:21.196729 7109 raft_consensus.cc:3037] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:21.197113 7005 raft_consensus.cc:2454] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6079fbfc3ef84e149e87316f28948c85 in term 0.
I20260807 03:02:21.199157 7109 raft_consensus.cc:504] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.199842 7109 leader_election.cc:288] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [CANDIDATE]: Term 1 election: Requested vote from peers a630f9b3c4814607ac398df889f6a0b2 (127.5.122.2:40925), 54ac1b9a849d49f199898b356fbada98 (127.5.122.3:36205)
I20260807 03:02:21.200191 6820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4179f48d374276b5c73e18811281eb" candidate_uuid: "6079fbfc3ef84e149e87316f28948c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a630f9b3c4814607ac398df889f6a0b2"
I20260807 03:02:21.200424 6820 raft_consensus.cc:3037] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:21.200629 7005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4179f48d374276b5c73e18811281eb" candidate_uuid: "6079fbfc3ef84e149e87316f28948c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ac1b9a849d49f199898b356fbada98"
I20260807 03:02:21.200804 7005 raft_consensus.cc:3037] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:21.202778 6820 raft_consensus.cc:2454] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6079fbfc3ef84e149e87316f28948c85 in term 1.
I20260807 03:02:21.203125 6533 leader_election.cc:302] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6079fbfc3ef84e149e87316f28948c85, a630f9b3c4814607ac398df889f6a0b2; no voters:
I20260807 03:02:21.203155 7005 raft_consensus.cc:2454] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6079fbfc3ef84e149e87316f28948c85 in term 1.
I20260807 03:02:21.203388 7109 raft_consensus.cc:2781] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:21.203859 7109 raft_consensus.cc:686] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [term 1 LEADER]: Becoming Leader. State: Replica: 6079fbfc3ef84e149e87316f28948c85, State: Running, Role: LEADER
I20260807 03:02:21.204440 7109 consensus_queue.cc:237] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } }
I20260807 03:02:21.209972 6413 catalog_manager.cc:5495] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6079fbfc3ef84e149e87316f28948c85 (127.5.122.1). New cstate: current_term: 1 leader_uuid: "6079fbfc3ef84e149e87316f28948c85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6079fbfc3ef84e149e87316f28948c85" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a630f9b3c4814607ac398df889f6a0b2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ac1b9a849d49f199898b356fbada98" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36205 } health_report { overall_health: UNKNOWN } } }
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:21.235965 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.221924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.5.122.1:44101)
0807 03:02:21.222107 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.222113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.222562 (+ 449us) server_negotiation.cc:407] Connection header received
0807 03:02:21.222727 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.222731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.222807 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.222897 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.224397 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.225127 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.226381 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.226768 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.226886 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.230727 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.230738 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.230753 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.230785 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.233322 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.233933 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.233940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.233942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.234089 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.234489 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.234493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.234495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.234832 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.235117 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.235422 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.235648 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071741, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071741, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
W20260807 03:02:21.277550 7075 tablet.cc:2369] T cb4179f48d374276b5c73e18811281eb P 54ac1b9a849d49f199898b356fbada98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:02:21.283032 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.269989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.5.122.1:44101)
0807 03:02:21.270226 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.270232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.270528 (+ 296us) server_negotiation.cc:407] Connection header received
0807 03:02:21.270673 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.270677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.270758 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.270850 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.272131 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.272953 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.274219 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.274724 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.274822 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.278660 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.278665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.278669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.278702 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.280958 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.281483 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.281486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.281488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.281602 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.281917 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.281919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.281921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.282181 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.282426 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.282598 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.282769 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260807 03:02:21.311268 6882 tablet.cc:2369] T cb4179f48d374276b5c73e18811281eb P a630f9b3c4814607ac398df889f6a0b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
W20260807 03:02:21.331306 6696 tablet.cc:2369] T cb4179f48d374276b5c73e18811281eb P 6079fbfc3ef84e149e87316f28948c85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:02:21.332614 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.317854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49224 (local address 127.5.122.1:44101)
0807 03:02:21.318094 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.318103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.318571 (+ 468us) server_negotiation.cc:407] Connection header received
0807 03:02:21.319032 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.319039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.319169 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.319281 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.320881 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.321788 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.323076 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.323535 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.323638 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.327314 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.327320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.327323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.327357 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.330064 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.330757 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.330761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.330764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.330896 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.331282 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.331285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.331292 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.331626 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.331847 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.332142 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.332328 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 03:02:21.334798 6575 tablet_service.cc:2908] Rejecting scan request for tablet cb4179f48d374276b5c73e18811281eb: Uninitialized: clean time has not yet been initialized
I20260807 03:02:21.339121 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6507
I20260807 03:02:21.351279 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.339888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.5.122.1:44101)
0807 03:02:21.340039 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.340046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.340458 (+ 412us) server_negotiation.cc:407] Connection header received
0807 03:02:21.340581 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.340585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.340662 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.340744 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.341806 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.342570 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.343879 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.344315 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.344413 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.346457 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.346462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.346465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.346497 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.349105 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.349659 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.349663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.349665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.349782 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.350043 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.350046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.350048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.350369 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.350564 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.350812 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.350977 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 03:02:21.351955 6687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:21.442302 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6507
I20260807 03:02:21.463495 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6702
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:02:21.478005 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.464339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.5.122.2:40925)
0807 03:02:21.464497 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.464501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.464518 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:02:21.464572 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.464576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.464661 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.464740 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.466077 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.466864 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.468160 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.468562 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.468655 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.472691 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.472704 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.472718 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.472748 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.475568 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.476179 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.476185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.476188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.476299 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.476617 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.476620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.476622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.476981 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.477237 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.477510 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.477671 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:02:21.478883 6873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:21.575953 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6702
I20260807 03:02:21.597476 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6891
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:02:21.611370 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:21.598371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.5.122.3:36205)
0807 03:02:21.598538 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:21.598543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:21.598561 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:02:21.598653 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:21.598657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:21.598721 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:21.598801 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:21.599993 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.600717 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.601891 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:21.602284 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:21.602443 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:21.606170 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:21.606186 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:21.606203 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:21.606235 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:21.608918 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.609447 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.609453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.609455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.609578 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:21.609851 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:21.609854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:21.609856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:21.610191 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:21.610460 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:21.610891 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 03:02:21.611052 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:02:21.612023 7067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:21.698166 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6891
I20260807 03:02:21.717684 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6365
I20260807 03:02:21.718849 6455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:21.823748 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6365
2026-08-07T03:02:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4041 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[7147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[7147](info): set up 2 sockets
Aug 07 03:02:21 dist-test-slave-fgb3 krb5kdc[7147](info): commencing operation
krb5kdc: starting...
W20260807 03:02:22.992347 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:23 dist-test-slave-fgb3 krb5kdc[7147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:23.174690 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:46261
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:46261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:23.494588 7165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:23.494879 7165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:23.495009 7165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:23.502086 7165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:23.502202 7165 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 03:02:23.502250 7165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:23.502349 7165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:23.502424 7165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:23.502502 7165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:23.512379 7165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:46261
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:46261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:23.513631 7165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:23.515043 7165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:23.526429 7175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:23.526520 7176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:23.526712 7178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:23.527673 7165 server_base.cc:1034] running on GCE node
I20260807 03:02:23.528386 7165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:23.529748 7165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:23.530951 7165 hybrid_clock.cc:648] HybridClock initialized: now 1786071743530876 us; error 53 us; skew 500 ppm
I20260807 03:02:23.531260 7165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:23 dist-test-slave-fgb3 krb5kdc[7147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071743, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:23.534299 7165 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:23.535734 7165 webserver.cc:460] Webserver started at http://127.5.122.62:41575/ using document root <none> and password file <none>
I20260807 03:02:23.536408 7165 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:23.536511 7165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:23.536823 7165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:23.539796 7165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "53fa177cdcf743618c9037095ba6955e"
format_stamp: "Formatted at 2026-08-07 03:02:23 on dist-test-slave-fgb3"
server_key: "19e21493b93d2d445f5a51c604ef6d81"
server_key_iv: "0ab9e43b7a13b6f11d503915f5efb6ff"
server_key_version: "encryptionkey@0"
I20260807 03:02:23.540582 7165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "53fa177cdcf743618c9037095ba6955e"
format_stamp: "Formatted at 2026-08-07 03:02:23 on dist-test-slave-fgb3"
server_key: "19e21493b93d2d445f5a51c604ef6d81"
server_key_iv: "0ab9e43b7a13b6f11d503915f5efb6ff"
server_key_version: "encryptionkey@0"
I20260807 03:02:23.545640 7165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 03:02:23.549014 7187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:23.550297 7165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:23.550529 7165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "53fa177cdcf743618c9037095ba6955e"
format_stamp: "Formatted at 2026-08-07 03:02:23 on dist-test-slave-fgb3"
server_key: "19e21493b93d2d445f5a51c604ef6d81"
server_key_iv: "0ab9e43b7a13b6f11d503915f5efb6ff"
server_key_version: "encryptionkey@0"
I20260807 03:02:23.550731 7165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:23.567456 7165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:23.571800 7165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:23.572070 7165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:23.592967 7165 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:46261
I20260807 03:02:23.592979 7276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:46261 every 8 connection(s)
I20260807 03:02:23.594352 7165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:23.596697 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 7165
I20260807 03:02:23.596940 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:23.597792 5608 external_mini_cluster.cc:1428] Setting key 33c83eb99317076e75707bec2ec547ab
I20260807 03:02:23.598726 7279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 03:02:23 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:23.612955 7279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e: Bootstrap starting.
I20260807 03:02:23.616429 7279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:23.617515 7279 log.cc:824] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:23.620411 7279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e: No bootstrap required, opened a new log
I20260807 03:02:23.620617 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:23.599858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.5.122.62:46261)
0807 03:02:23.600599 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:23.600611 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:23.600651 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:02:23.602170 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:23.602190 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:23.602622 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:23.602953 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:23.604765 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:23.606216 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:23.607883 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:23.608553 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:23.608695 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:23.613556 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:23.613564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:23.613578 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:23.613616 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:23.616549 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:23.617283 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:23.617291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:23.617297 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:23.617470 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:23.617870 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:23.617873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:23.617875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:23.618517 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:23.618811 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:23.619234 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 03:02:23.619643 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":346,"threads_started":1}
I20260807 03:02:23.628182 7279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53fa177cdcf743618c9037095ba6955e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 46261 } }
I20260807 03:02:23.628530 7279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:23.628623 7279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fa177cdcf743618c9037095ba6955e, State: Initialized, Role: FOLLOWER
I20260807 03:02:23.629136 7279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53fa177cdcf743618c9037095ba6955e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 46261 } }
I20260807 03:02:23.629348 7279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:23.629472 7279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:23.629628 7279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:23.631804 7279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53fa177cdcf743618c9037095ba6955e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 46261 } }
I20260807 03:02:23.632284 7279 leader_election.cc:302] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53fa177cdcf743618c9037095ba6955e; no voters:
I20260807 03:02:23.632867 7279 leader_election.cc:288] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:23.632968 7287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:23.633599 7287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [term 1 LEADER]: Becoming Leader. State: Replica: 53fa177cdcf743618c9037095ba6955e, State: Running, Role: LEADER
I20260807 03:02:23.634183 7287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53fa177cdcf743618c9037095ba6955e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 46261 } }
I20260807 03:02:23.634984 7279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:23.637038 7288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53fa177cdcf743618c9037095ba6955e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53fa177cdcf743618c9037095ba6955e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 46261 } } }
I20260807 03:02:23.637322 7288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:23.639354 7296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:23.639161 7290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53fa177cdcf743618c9037095ba6955e. Latest consensus state: current_term: 1 leader_uuid: "53fa177cdcf743618c9037095ba6955e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53fa177cdcf743618c9037095ba6955e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 46261 } } }
I20260807 03:02:23.639483 7290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:23.643958 7296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:23.652699 7296 catalog_manager.cc:1347] Generated new cluster ID: ebb1fcac47004a80be6e29398c7c159b
I20260807 03:02:23.652817 7296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:23.666923 7296 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:23.668363 7296 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:23.676903 7296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e: Generated new TSK 0
I20260807 03:02:23.677795 7296 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:02:23.728863 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:46261
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:24.042629 7320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:24.042913 7320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:24.043044 7320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:24.050218 7320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:24.050336 7320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:24.050467 7320 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:02:24.050585 7320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:24.060307 7320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:46261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:24.061548 7320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:24.062892 7320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:24.075589 7328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:24.075616 7327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:24.076076 7330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:24.077107 7320 server_base.cc:1034] running on GCE node
I20260807 03:02:24.077929 7320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:24.078917 7320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:24.080142 7320 hybrid_clock.cc:648] HybridClock initialized: now 1786071744080064 us; error 49 us; skew 500 ppm
I20260807 03:02:24.080431 7320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:24 dist-test-slave-fgb3 krb5kdc[7147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071744, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:24.083638 7320 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:24.085125 7320 webserver.cc:460] Webserver started at http://127.5.122.1:41583/ using document root <none> and password file <none>
I20260807 03:02:24.085855 7320 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:24.085960 7320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:24.086277 7320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:24.089277 7320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "27a94e3d559d4db1b35a1d62c37bc815"
format_stamp: "Formatted at 2026-08-07 03:02:24 on dist-test-slave-fgb3"
server_key: "15f8530861682f94368fe54e05a45451"
server_key_iv: "31b90d17eb0b6937059fb44748652f1d"
server_key_version: "encryptionkey@0"
I20260807 03:02:24.090085 7320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "27a94e3d559d4db1b35a1d62c37bc815"
format_stamp: "Formatted at 2026-08-07 03:02:24 on dist-test-slave-fgb3"
server_key: "15f8530861682f94368fe54e05a45451"
server_key_iv: "31b90d17eb0b6937059fb44748652f1d"
server_key_version: "encryptionkey@0"
I20260807 03:02:24.095383 7320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 03:02:24.098886 7343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:24.100181 7320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:02:24.100344 7320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "27a94e3d559d4db1b35a1d62c37bc815"
format_stamp: "Formatted at 2026-08-07 03:02:24 on dist-test-slave-fgb3"
server_key: "15f8530861682f94368fe54e05a45451"
server_key_iv: "31b90d17eb0b6937059fb44748652f1d"
server_key_version: "encryptionkey@0"
I20260807 03:02:24.100567 7320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:24.115247 7320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:24.119297 7320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:24.119557 7320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:24.122401 7320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:24.122493 7320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:24.122648 7320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:24.122749 7320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:02:24 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071744, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:24.146965 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.123441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:38251 (local address 127.5.122.62:46261)
0807 03:02:24.123846 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:24.123850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:24.125030 (+ 1180us) server_negotiation.cc:407] Connection header received
0807 03:02:24.127172 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:24.127177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:24.127271 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:24.127373 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:24.130075 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.131042 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.133927 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.134538 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.134682 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.139953 (+ 5271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:24.139960 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:24.139965 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:24.140009 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:24.143341 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.144327 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.144331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.144333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.144455 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.144995 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.144998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.145000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.145301 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:24.145449 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:24.146433 (+ 984us) server_negotiation.cc:299] Negotiation successful
0807 03:02:24.146690 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":213,"threads_started":1}
I20260807 03:02:24.148577 7361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.124048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:46261 (local address 127.5.122.1:38251)
0807 03:02:24.124877 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:24.124914 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:24.126880 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:24.127835 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:24.127847 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:24.128806 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:24.129856 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.129890 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.131220 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.131226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.133750 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.133769 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.134841 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.134848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.135217 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.136073 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:24.136108 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:24.139626 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:24.143508 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.143520 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.143544 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.144201 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.144561 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.144566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.144570 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.144896 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.145454 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:24.145463 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:24.145962 (+ 499us) client_negotiation.cc:769] Sending connection context
0807 03:02:24.146288 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 03:02:24.146961 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":385,"threads_started":1}
I20260807 03:02:24.168462 7320 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:43605
I20260807 03:02:24.168502 7529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:43605 every 8 connection(s)
I20260807 03:02:24.169162 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.155093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:50415 (local address 127.5.122.62:46261)
0807 03:02:24.155254 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:24.155259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:24.155861 (+ 602us) server_negotiation.cc:407] Connection header received
0807 03:02:24.156038 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:24.156044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:24.156138 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:24.156230 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:24.157156 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.158131 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.160230 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.160802 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.160936 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.163581 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:24.163585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:24.163589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:24.163619 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:24.166454 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.167138 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.167144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.167147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.167331 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.167732 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.167737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.167741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.168020 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:24.168150 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:24.168593 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 03:02:24.168818 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 03:02:24.169255 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.155108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:46261 (local address 127.5.122.1:50415)
0807 03:02:24.155759 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:24.155777 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:24.155919 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:24.156255 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:24.156259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:24.156729 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:24.157018 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.157027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.158282 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.158288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.160085 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.160102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.161203 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.161213 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.161389 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.162149 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:24.162174 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:24.163363 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:24.166633 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.166636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.166639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.167019 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.167456 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.167460 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.167463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.167623 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.168135 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:24.168139 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:24.168343 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 03:02:24.168476 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 03:02:24.168814 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":410,"threads_started":1}
I20260807 03:02:24.170011 7320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:24.172156 7530 heartbeater.cc:347] Connected to a master server at 127.5.122.62:46261
I20260807 03:02:24.172394 7530 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:24.173206 7530 heartbeater.cc:511] Master 127.5.122.62:46261 requested a full tablet report, sending...
I20260807 03:02:24.175096 7210 ts_manager.cc:194] Registered new tserver with Master: 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1:43605)
I20260807 03:02:24.176558 7210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:38251
I20260807 03:02:24.177067 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 7320
I20260807 03:02:24.177279 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:24.177883 5608 external_mini_cluster.cc:1428] Setting key 3fd279224b4205be1ca5cf642f8e7e7b
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:02:24.217509 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:46261
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:24.530490 7535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:24.530772 7535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:24.530905 7535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:24.538087 7535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:24.538205 7535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:24.538353 7535 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:02:24.538471 7535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:24.547958 7535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:46261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:24.549165 7535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:24.550495 7535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:24.562985 7546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:24.563016 7549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:24.563053 7547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:24.564141 7535 server_base.cc:1034] running on GCE node
I20260807 03:02:24.564770 7535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:24.565639 7535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:24.566854 7535 hybrid_clock.cc:648] HybridClock initialized: now 1786071744566788 us; error 40 us; skew 500 ppm
I20260807 03:02:24.567142 7535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:24 dist-test-slave-fgb3 krb5kdc[7147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071744, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:24.574908 7535 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:24.576262 7535 webserver.cc:460] Webserver started at http://127.5.122.2:44363/ using document root <none> and password file <none>
I20260807 03:02:24.576903 7535 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:24.576988 7535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:24.577260 7535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:24.580097 7535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "858bf8708057484f84ee1e61946b73df"
format_stamp: "Formatted at 2026-08-07 03:02:24 on dist-test-slave-fgb3"
server_key: "2ce7303fcffde9de20913ad953be3fef"
server_key_iv: "e9695073067d9f6fe807c0ce36e07fbb"
server_key_version: "encryptionkey@0"
I20260807 03:02:24.580834 7535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "858bf8708057484f84ee1e61946b73df"
format_stamp: "Formatted at 2026-08-07 03:02:24 on dist-test-slave-fgb3"
server_key: "2ce7303fcffde9de20913ad953be3fef"
server_key_iv: "e9695073067d9f6fe807c0ce36e07fbb"
server_key_version: "encryptionkey@0"
I20260807 03:02:24.585669 7535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:02:24.588932 7558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:24.590119 7535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:24.590253 7535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "858bf8708057484f84ee1e61946b73df"
format_stamp: "Formatted at 2026-08-07 03:02:24 on dist-test-slave-fgb3"
server_key: "2ce7303fcffde9de20913ad953be3fef"
server_key_iv: "e9695073067d9f6fe807c0ce36e07fbb"
server_key_version: "encryptionkey@0"
I20260807 03:02:24.590492 7535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:24.628521 7535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:24.632504 7535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:24.632771 7535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:24.635617 7535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:24.635715 7535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:24.635870 7535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:24.635929 7535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:02:24 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071744, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:24.657894 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.635910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:39649 (local address 127.5.122.62:46261)
0807 03:02:24.636076 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:24.636082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:24.637566 (+ 1484us) server_negotiation.cc:407] Connection header received
0807 03:02:24.639603 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:24.639609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:24.639715 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:24.639803 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:24.641846 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.642922 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.645382 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.645906 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.646002 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.651283 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:24.651300 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:24.651305 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:24.651347 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:24.654523 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.655608 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.655614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.655618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.655749 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.656187 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.656192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.656196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.656492 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:24.656634 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:24.657301 (+ 667us) server_negotiation.cc:299] Negotiation successful
0807 03:02:24.657532 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 03:02:24.659077 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.636641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:46261 (local address 127.5.122.2:39649)
0807 03:02:24.637417 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:24.637454 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:24.639336 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:24.640046 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:24.640055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:24.640818 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:24.641644 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.641693 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.643129 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.643135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.645159 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.645175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.646157 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.646166 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.646499 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.647400 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:24.647438 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:24.650982 (+ 3544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:24.654724 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.654733 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.654752 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.655438 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.655862 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.655867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.655872 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.656079 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.656624 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:24.656632 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:24.656961 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 03:02:24.657195 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 03:02:24.657718 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":347,"threads_started":1}
I20260807 03:02:24.678572 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.664489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:51219 (local address 127.5.122.62:46261)
0807 03:02:24.664671 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:24.664677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:24.665243 (+ 566us) server_negotiation.cc:407] Connection header received
0807 03:02:24.665406 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:24.665410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:24.665466 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:24.665539 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:24.666910 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.667643 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.669641 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.670073 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.670182 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.672625 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:24.672632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:24.672637 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:24.672676 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:24.675650 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.676488 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.676493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.676497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.676624 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:24.677042 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:24.677047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:24.677049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:24.677465 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:24.677598 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:24.678004 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 03:02:24.678205 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:02:24.678732 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:24.664467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:46261 (local address 127.5.122.2:51219)
0807 03:02:24.665138 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:24.665160 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:24.665303 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:24.665580 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:24.665585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:24.666203 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:24.666684 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.666704 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.667779 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.667784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.669460 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:24.669476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:24.670252 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:24.670258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:24.670452 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:24.671264 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:24.671292 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:24.672368 (+ 1076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:24.675770 (+ 3402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.675775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.675779 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.676367 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.676736 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:24.676741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:24.676745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:24.676868 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:24.677586 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:24.677590 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:24.677757 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 03:02:24.677877 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 03:02:24.678206 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":350,"threads_started":1}
I20260807 03:02:24.686599 7535 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:38349
I20260807 03:02:24.686627 7751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:38349 every 8 connection(s)
I20260807 03:02:24.687726 7535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:24.689575 7752 heartbeater.cc:347] Connected to a master server at 127.5.122.62:46261
I20260807 03:02:24.689778 7752 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:24.690433 7752 heartbeater.cc:511] Master 127.5.122.62:46261 requested a full tablet report, sending...
I20260807 03:02:24.691632 7210 ts_manager.cc:194] Registered new tserver with Master: 858bf8708057484f84ee1e61946b73df (127.5.122.2:38349)
I20260807 03:02:24.692370 7210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:39649
I20260807 03:02:24.694415 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 7535
I20260807 03:02:24.694587 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:24.695029 5608 external_mini_cluster.cc:1428] Setting key 06cd1a15e5d7c3f40abb10f3799415c5
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:24.734457 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:46261
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:25.051681 7756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:25.051997 7756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:25.052152 7756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:25.059628 7756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:25.059762 7756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:25.059883 7756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:02:25.059996 7756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:25.070030 7756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:46261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:25.071373 7756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:25.072758 7756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:25.085400 7775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:25.085501 7769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:25.085542 7771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:25.086489 7756 server_base.cc:1034] running on GCE node
I20260807 03:02:25.087066 7756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:25.087996 7756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:25.089243 7756 hybrid_clock.cc:648] HybridClock initialized: now 1786071745089159 us; error 57 us; skew 500 ppm
I20260807 03:02:25.089548 7756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:25 dist-test-slave-fgb3 krb5kdc[7147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:25.092660 7756 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:25.094084 7756 webserver.cc:460] Webserver started at http://127.5.122.3:35317/ using document root <none> and password file <none>
I20260807 03:02:25.094776 7756 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:25.094867 7756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:25.095145 7756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:25.098032 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "5042ba7deee74f28abc2984c8970d487"
format_stamp: "Formatted at 2026-08-07 03:02:25 on dist-test-slave-fgb3"
server_key: "2cbc964d17019e08b4ed16e15725e505"
server_key_iv: "db2a0e47290c0b87734ab8203cd315f2"
server_key_version: "encryptionkey@0"
I20260807 03:02:25.098878 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "5042ba7deee74f28abc2984c8970d487"
format_stamp: "Formatted at 2026-08-07 03:02:25 on dist-test-slave-fgb3"
server_key: "2cbc964d17019e08b4ed16e15725e505"
server_key_iv: "db2a0e47290c0b87734ab8203cd315f2"
server_key_version: "encryptionkey@0"
I20260807 03:02:25.103801 7756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:02:25.107215 7788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:25.108542 7756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:02:25.108700 7756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "5042ba7deee74f28abc2984c8970d487"
format_stamp: "Formatted at 2026-08-07 03:02:25 on dist-test-slave-fgb3"
server_key: "2cbc964d17019e08b4ed16e15725e505"
server_key_iv: "db2a0e47290c0b87734ab8203cd315f2"
server_key_version: "encryptionkey@0"
I20260807 03:02:25.108898 7756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:25.152899 7756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:25.157078 7756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:25.157320 7756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:25.160192 7756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:25.160300 7756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:25.160449 7756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:25.160528 7756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:02:25 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:25.179304 7530 heartbeater.cc:503] Master 127.5.122.62:46261 was elected leader, sending a full tablet report...
I20260807 03:02:25.182915 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.160727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:59445 (local address 127.5.122.62:46261)
0807 03:02:25.160951 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.160957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.162597 (+ 1640us) server_negotiation.cc:407] Connection header received
0807 03:02:25.164592 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.164598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.164718 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.164824 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:25.167253 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.168183 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.170573 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.171269 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.171400 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.176204 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:25.176208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:25.176212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:25.176245 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:25.178864 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.179997 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.180002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.180005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.180180 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.180757 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.180773 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.180777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.181133 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:25.181270 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.182183 (+ 913us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.182507 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:02:25.183907 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.161540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:46261 (local address 127.5.122.3:59445)
0807 03:02:25.162447 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.162488 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.164313 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.165108 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.165124 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.166059 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:25.167042 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.167075 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.168343 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.168348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.170387 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.170402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.171525 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.171533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.171836 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.172657 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:25.172689 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:25.175976 (+ 3287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:25.179013 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:25.179022 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:25.179039 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:25.179668 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:25.180326 (+ 658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:25.180331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:25.180334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:25.180595 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:25.181259 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:25.181279 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:25.181747 (+ 468us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.182034 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.182611 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":387,"threads_started":1}
I20260807 03:02:25.204342 7756 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:39815
I20260807 03:02:25.204559 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.191024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:58821 (local address 127.5.122.62:46261)
0807 03:02:25.191217 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.191223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.191868 (+ 645us) server_negotiation.cc:407] Connection header received
0807 03:02:25.192034 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.192039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.192114 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.192191 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:25.193512 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.194367 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.196092 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.196497 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.196604 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.198828 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:25.198834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:25.198839 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:25.198877 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:25.201954 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.202696 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.202701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.202705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.202842 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.203183 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.203188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.203191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.203460 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:25.203588 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.204003 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.204221 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:02:25.204869 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.191081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:46261 (local address 127.5.122.3:58821)
0807 03:02:25.191758 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.191789 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.191928 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.192238 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.192243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.192927 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:25.193380 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.193398 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.194501 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.194506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.195962 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.195978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.196697 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.196707 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.196834 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.197525 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:25.197541 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:25.198653 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:25.202106 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:25.202112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:25.202116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:25.202590 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:25.202949 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:25.202954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:25.202958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:25.203088 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:25.203575 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:25.203580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:25.203761 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.203892 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.204275 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":366,"threads_started":1}
I20260807 03:02:25.204430 7957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:39815 every 8 connection(s)
I20260807 03:02:25.205963 7756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:25.208317 7960 heartbeater.cc:347] Connected to a master server at 127.5.122.62:46261
I20260807 03:02:25.208547 7960 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:25.209143 7960 heartbeater.cc:511] Master 127.5.122.62:46261 requested a full tablet report, sending...
I20260807 03:02:25.210480 7210 ts_manager.cc:194] Registered new tserver with Master: 5042ba7deee74f28abc2984c8970d487 (127.5.122.3:39815)
I20260807 03:02:25.210505 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 7756
I20260807 03:02:25.210693 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:25.211236 5608 external_mini_cluster.cc:1428] Setting key 0696bc673d2bb4229ec73ccb7d0fcf2f
I20260807 03:02:25.211601 7210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:59445
I20260807 03:02:25.214118 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 03:02:25 dist-test-slave-fgb3 krb5kdc[7147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 03:02:25 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:25.259824 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.246948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.5.122.62:46261)
0807 03:02:25.247155 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.247161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.247314 (+ 153us) server_negotiation.cc:407] Connection header received
0807 03:02:25.247509 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.247513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.247575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.247656 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:25.249044 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.249753 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.251080 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.251573 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.251679 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.255365 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:25.255369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:25.255372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:25.255404 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:25.257581 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.258124 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.258128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.258130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.258238 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.258572 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.258575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.258577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.258847 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:25.259023 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.259323 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.259509 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:02:25.263844 7210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:25.270670 7210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:25.292495 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.282387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:38349 (local address 127.0.0.1:32870)
0807 03:02:25.283688 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.283735 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.283906 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.284728 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.284738 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.284771 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.285167 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.285180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.287274 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.287280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.289643 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.289660 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.291505 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.291512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.291747 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.291801 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.291903 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.292098 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":410,"threads_started":1}
I20260807 03:02:25.293818 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.282548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.5.122.2:38349)
0807 03:02:25.283427 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.283437 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.284115 (+ 678us) server_negotiation.cc:407] Connection header received
0807 03:02:25.284224 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.284232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.284489 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.285399 (+ 910us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.285607 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.287117 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.289864 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.291274 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.292977 (+ 1703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.293204 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.293339 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.293523 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":568,"threads_started":1}
I20260807 03:02:25.303579 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.282858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39815 (local address 127.0.0.1:43194)
0807 03:02:25.292719 (+ 9861us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.292736 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.292823 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.296359 (+ 3536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.296365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.296394 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.296564 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.296573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.299392 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.299398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.301123 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.301134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.302774 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.302780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.302973 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.303005 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.303101 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.303235 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9759,"mutex_wait_us":1,"thread_start_us":16933,"threads_started":1}
I20260807 03:02:25.303668 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.283390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.5.122.3:39815)
0807 03:02:25.295686 (+ 12296us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.295695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.295733 (+ 38us) server_negotiation.cc:407] Connection header received
0807 03:02:25.295833 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.295842 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.296082 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.296335 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.296706 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.299214 (+ 2508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.301270 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.302628 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.302741 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.302978 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.303153 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.303299 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12107,"thread_start_us":14963,"threads_started":1}
I20260807 03:02:25.303372 7639 tablet_service.cc:1467] Processing CreateTablet for tablet 827efe3b93564266adaf4d2e84fa20f8 (DEFAULT_TABLE table=test-table [id=0a07c6e9df7c41a7940b088aa017c0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:25.305037 7639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827efe3b93564266adaf4d2e84fa20f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.307144 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.283916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43605 (local address 127.0.0.1:54234)
0807 03:02:25.298781 (+ 14865us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.298802 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.298924 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.300080 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.300085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.300109 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.300513 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.300525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.302370 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.302374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.304571 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.304587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.306384 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.306391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.306559 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.306591 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.306686 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.306803 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14715,"mutex_wait_us":1,"thread_start_us":14550,"threads_started":1}
I20260807 03:02:25.307194 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.284449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.5.122.1:43605)
0807 03:02:25.298940 (+ 14491us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.298948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.298978 (+ 30us) server_negotiation.cc:407] Connection header received
0807 03:02:25.299070 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.299077 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.299313 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.299500 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.300693 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.302179 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.304757 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.306153 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.306284 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.306494 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.306743 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.306913 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14322,"thread_start_us":15547,"threads_started":1}
I20260807 03:02:25.307029 7851 tablet_service.cc:1467] Processing CreateTablet for tablet 827efe3b93564266adaf4d2e84fa20f8 (DEFAULT_TABLE table=test-table [id=0a07c6e9df7c41a7940b088aa017c0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:25.308223 7851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827efe3b93564266adaf4d2e84fa20f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.310854 7422 tablet_service.cc:1467] Processing CreateTablet for tablet 827efe3b93564266adaf4d2e84fa20f8 (DEFAULT_TABLE table=test-table [id=0a07c6e9df7c41a7940b088aa017c0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:25.312079 7422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827efe3b93564266adaf4d2e84fa20f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.323156 7985 tablet_bootstrap.cc:492] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Bootstrap starting.
I20260807 03:02:25.325894 7986 tablet_bootstrap.cc:492] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Bootstrap starting.
I20260807 03:02:25.327481 7984 tablet_bootstrap.cc:492] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Bootstrap starting.
I20260807 03:02:25.328981 7986 tablet_bootstrap.cc:654] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.329937 7986 log.cc:824] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:25.330003 7984 tablet_bootstrap.cc:654] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.330940 7984 log.cc:824] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:25.331833 7985 tablet_bootstrap.cc:654] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.332599 7986 tablet_bootstrap.cc:492] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: No bootstrap required, opened a new log
I20260807 03:02:25.332835 7986 ts_tablet_manager.cc:1397] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260807 03:02:25.332826 7985 log.cc:824] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:25.334059 7984 tablet_bootstrap.cc:492] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: No bootstrap required, opened a new log
I20260807 03:02:25.334380 7984 ts_tablet_manager.cc:1397] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 03:02:25.335781 7985 tablet_bootstrap.cc:492] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: No bootstrap required, opened a new log
I20260807 03:02:25.336102 7985 ts_tablet_manager.cc:1397] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260807 03:02:25.340474 7986 raft_consensus.cc:348] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.340788 7986 raft_consensus.cc:374] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.340889 7986 raft_consensus.cc:729] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a94e3d559d4db1b35a1d62c37bc815, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.341437 7986 consensus_queue.cc:260] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.343036 7986 ts_tablet_manager.cc:1428] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260807 03:02:25.344583 7984 raft_consensus.cc:348] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.345018 7984 raft_consensus.cc:374] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.345108 7984 raft_consensus.cc:729] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858bf8708057484f84ee1e61946b73df, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.345811 7984 consensus_queue.cc:260] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.346647 7985 raft_consensus.cc:348] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.347060 7985 raft_consensus.cc:374] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.347172 7985 raft_consensus.cc:729] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5042ba7deee74f28abc2984c8970d487, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.347478 7984 ts_tablet_manager.cc:1428] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260807 03:02:25.347834 7752 heartbeater.cc:503] Master 127.5.122.62:46261 was elected leader, sending a full tablet report...
I20260807 03:02:25.347725 7985 consensus_queue.cc:260] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.348803 7960 heartbeater.cc:503] Master 127.5.122.62:46261 was elected leader, sending a full tablet report...
I20260807 03:02:25.349339 7985 ts_tablet_manager.cc:1428] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Time spent starting tablet: real 0.013s user 0.003s sys 0.010s
I20260807 03:02:25.415187 7992 raft_consensus.cc:482] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:25.415515 7992 raft_consensus.cc:504] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.417148 7992 leader_election.cc:288] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1:43605), 5042ba7deee74f28abc2984c8970d487 (127.5.122.3:39815)
W20260807 03:02:25.422691 7531 tablet.cc:2369] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:02:25.423810 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.417719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:46041 (local address 127.5.122.3:39815)
0807 03:02:25.417942 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.417949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.418587 (+ 638us) server_negotiation.cc:407] Connection header received
0807 03:02:25.418751 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.418756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.418811 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.418883 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.419511 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.420191 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.421960 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.422759 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.422872 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.422973 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.423432 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.423562 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:02:25.423966 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.417454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:33289 (local address 127.5.122.1:43605)
0807 03:02:25.417660 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.417666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.418421 (+ 755us) server_negotiation.cc:407] Connection header received
0807 03:02:25.418680 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.418685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.418757 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.418831 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.419426 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.420121 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.421976 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.423024 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.423161 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.423251 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.423629 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.423756 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 03:02:25.424034 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.417513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43605 (local address 127.5.122.2:33289)
0807 03:02:25.418298 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.418334 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.418500 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.418862 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.418867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.418892 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.419287 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.419299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.420238 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.420244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.421830 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.421843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.423236 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.423242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.423434 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.423465 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.423586 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.423708 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":100,"thread_start_us":456,"threads_started":1}
I20260807 03:02:25.424223 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.417610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39815 (local address 127.5.122.2:46041)
0807 03:02:25.418496 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.418511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.418618 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.418911 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.418915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.418940 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.419363 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.419373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.420329 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.420333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.421802 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.421818 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.423061 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.423066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.423221 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.423274 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.423371 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.423493 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":41,"thread_start_us":545,"threads_started":1}
I20260807 03:02:25.425207 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827efe3b93564266adaf4d2e84fa20f8" candidate_uuid: "858bf8708057484f84ee1e61946b73df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a94e3d559d4db1b35a1d62c37bc815" is_pre_election: true
I20260807 03:02:25.425208 7888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827efe3b93564266adaf4d2e84fa20f8" candidate_uuid: "858bf8708057484f84ee1e61946b73df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5042ba7deee74f28abc2984c8970d487" is_pre_election: true
I20260807 03:02:25.425767 7449 raft_consensus.cc:2454] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 858bf8708057484f84ee1e61946b73df in term 0.
I20260807 03:02:25.425767 7888 raft_consensus.cc:2454] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 858bf8708057484f84ee1e61946b73df in term 0.
I20260807 03:02:25.426476 7559 leader_election.cc:302] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a94e3d559d4db1b35a1d62c37bc815, 858bf8708057484f84ee1e61946b73df; no voters:
I20260807 03:02:25.426959 7992 raft_consensus.cc:2781] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:25.427076 7992 raft_consensus.cc:482] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:25.427145 7992 raft_consensus.cc:3037] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.429268 7992 raft_consensus.cc:504] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.429795 7992 leader_election.cc:288] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [CANDIDATE]: Term 1 election: Requested vote from peers 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1:43605), 5042ba7deee74f28abc2984c8970d487 (127.5.122.3:39815)
I20260807 03:02:25.430168 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827efe3b93564266adaf4d2e84fa20f8" candidate_uuid: "858bf8708057484f84ee1e61946b73df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a94e3d559d4db1b35a1d62c37bc815"
I20260807 03:02:25.430256 7888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827efe3b93564266adaf4d2e84fa20f8" candidate_uuid: "858bf8708057484f84ee1e61946b73df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5042ba7deee74f28abc2984c8970d487"
I20260807 03:02:25.430369 7449 raft_consensus.cc:3037] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.430485 7888 raft_consensus.cc:3037] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.432524 7449 raft_consensus.cc:2454] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 858bf8708057484f84ee1e61946b73df in term 1.
I20260807 03:02:25.432654 7888 raft_consensus.cc:2454] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 858bf8708057484f84ee1e61946b73df in term 1.
I20260807 03:02:25.432907 7559 leader_election.cc:302] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a94e3d559d4db1b35a1d62c37bc815, 858bf8708057484f84ee1e61946b73df; no voters:
I20260807 03:02:25.433146 7992 raft_consensus.cc:2781] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:25.433525 7992 raft_consensus.cc:686] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 1 LEADER]: Becoming Leader. State: Replica: 858bf8708057484f84ee1e61946b73df, State: Running, Role: LEADER
I20260807 03:02:25.433950 7992 consensus_queue.cc:237] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.438299 7210 catalog_manager.cc:5495] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df reported cstate change: term changed from 0 to 1, leader changed from <none> to 858bf8708057484f84ee1e61946b73df (127.5.122.2). New cstate: current_term: 1 leader_uuid: "858bf8708057484f84ee1e61946b73df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } health_report { overall_health: UNKNOWN } } }
W20260807 03:02:25.439522 7753 tablet.cc:2369] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:02:25.458587 7961 tablet.cc:2369] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:02:25.479238 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.474131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32872 (local address 127.5.122.2:38349)
0807 03:02:25.474305 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.474327 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.474505 (+ 178us) server_negotiation.cc:407] Connection header received
0807 03:02:25.474769 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.474773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.474850 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.474973 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:25.475282 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.475988 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.477230 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.477654 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.477747 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.478164 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:25.478750 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:25.478861 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.478951 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.479022 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 03:02:25.488509 7888 raft_consensus.cc:1261] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Refusing update from remote peer 858bf8708057484f84ee1e61946b73df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:25.489786 7449 raft_consensus.cc:1261] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Refusing update from remote peer 858bf8708057484f84ee1e61946b73df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:25.489691 7998 consensus_queue.cc:1035] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:25.490646 7992 consensus_queue.cc:1035] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:25.522719 7210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:25.523083 7210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:25.527179 7210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e: Sending DeleteTablet for 3 replicas of tablet 827efe3b93564266adaf4d2e84fa20f8
I20260807 03:02:25.528445 7639 tablet_service.cc:1514] Processing DeleteTablet for tablet 827efe3b93564266adaf4d2e84fa20f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:25 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:32870
I20260807 03:02:25.528543 7851 tablet_service.cc:1514] Processing DeleteTablet for tablet 827efe3b93564266adaf4d2e84fa20f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:25 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:43194
I20260807 03:02:25.528537 7422 tablet_service.cc:1514] Processing DeleteTablet for tablet 827efe3b93564266adaf4d2e84fa20f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:25 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:54234
I20260807 03:02:25.533620 7210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:25.534778 7210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:25.541891 8015 tablet_replica.cc:330] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: stopping tablet replica
I20260807 03:02:25.542646 8015 raft_consensus.cc:2229] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:25.543228 8015 raft_consensus.cc:2258] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:25.544435 8018 tablet_replica.cc:330] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: stopping tablet replica
I20260807 03:02:25.544876 7638 tablet_service.cc:1467] Processing CreateTablet for tablet 53d61f4f4d8e427bb51f8af3b3aaceed (DEFAULT_TABLE table=test-table [id=221c83a1df4e461794b367d10c982965]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:25.545588 7638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d61f4f4d8e427bb51f8af3b3aaceed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.545997 7850 tablet_service.cc:1467] Processing CreateTablet for tablet 53d61f4f4d8e427bb51f8af3b3aaceed (DEFAULT_TABLE table=test-table [id=221c83a1df4e461794b367d10c982965]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:25.546512 7850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d61f4f4d8e427bb51f8af3b3aaceed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.553535 7985 tablet_bootstrap.cc:492] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: Bootstrap starting.
I20260807 03:02:25.554534 7421 tablet_service.cc:1467] Processing CreateTablet for tablet 53d61f4f4d8e427bb51f8af3b3aaceed (DEFAULT_TABLE table=test-table [id=221c83a1df4e461794b367d10c982965]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:25.555238 7421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d61f4f4d8e427bb51f8af3b3aaceed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.555522 7985 tablet_bootstrap.cc:654] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.555711 7984 tablet_bootstrap.cc:492] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: Bootstrap starting.
I20260807 03:02:25.557412 7984 tablet_bootstrap.cc:654] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.558892 7985 tablet_bootstrap.cc:492] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: No bootstrap required, opened a new log
I20260807 03:02:25.559048 7985 ts_tablet_manager.cc:1397] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 03:02:25.560896 7985 raft_consensus.cc:348] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.561179 7985 raft_consensus.cc:374] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.561277 7985 raft_consensus.cc:729] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5042ba7deee74f28abc2984c8970d487, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.561594 7985 consensus_queue.cc:260] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.561924 7986 tablet_bootstrap.cc:492] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: Bootstrap starting.
I20260807 03:02:25.563482 7986 tablet_bootstrap.cc:654] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.565676 7985 ts_tablet_manager.cc:1428] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260807 03:02:25.544996 8018 raft_consensus.cc:2229] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:25.567193 8018 raft_consensus.cc:2258] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:25.568933 7986 tablet_bootstrap.cc:492] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: No bootstrap required, opened a new log
I20260807 03:02:25.569092 7986 ts_tablet_manager.cc:1397] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 03:02:25.569742 8015 ts_tablet_manager.cc:1905] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:25.569770 8018 ts_tablet_manager.cc:1905] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:25.569901 7986 raft_consensus.cc:348] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.571863 7986 raft_consensus.cc:374] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.571997 7986 raft_consensus.cc:729] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a94e3d559d4db1b35a1d62c37bc815, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.572268 7986 consensus_queue.cc:260] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.572796 7986 ts_tablet_manager.cc:1428] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260807 03:02:25.576647 8015 ts_tablet_manager.cc:1918] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:25.576916 8015 log.cc:1196] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/827efe3b93564266adaf4d2e84fa20f8
I20260807 03:02:25.577538 7984 tablet_bootstrap.cc:492] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: No bootstrap required, opened a new log
I20260807 03:02:25.577711 7984 ts_tablet_manager.cc:1397] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.012s
I20260807 03:02:25.577972 8019 tablet_replica.cc:330] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: stopping tablet replica
I20260807 03:02:25.578166 8015 ts_tablet_manager.cc:1939] T 827efe3b93564266adaf4d2e84fa20f8 P 5042ba7deee74f28abc2984c8970d487: Deleting consensus metadata
I20260807 03:02:25.578934 7984 raft_consensus.cc:348] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.579190 7984 raft_consensus.cc:374] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.579317 7984 raft_consensus.cc:729] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858bf8708057484f84ee1e61946b73df, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.579627 8018 ts_tablet_manager.cc:1918] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:25.579782 8018 log.cc:1196] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/827efe3b93564266adaf4d2e84fa20f8
I20260807 03:02:25.579638 7984 consensus_queue.cc:260] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.580302 8018 ts_tablet_manager.cc:1939] T 827efe3b93564266adaf4d2e84fa20f8 P 27a94e3d559d4db1b35a1d62c37bc815: Deleting consensus metadata
I20260807 03:02:25.580274 7192 catalog_manager.cc:4841] TS 5042ba7deee74f28abc2984c8970d487 (127.5.122.3:39815): tablet 827efe3b93564266adaf4d2e84fa20f8 (table test-table [id=0a07c6e9df7c41a7940b088aa017c0fa]) successfully deleted
I20260807 03:02:25.580948 7984 ts_tablet_manager.cc:1428] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:25.579157 8019 raft_consensus.cc:2229] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:25.581811 7190 catalog_manager.cc:4841] TS 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1:43605): tablet 827efe3b93564266adaf4d2e84fa20f8 (table test-table [id=0a07c6e9df7c41a7940b088aa017c0fa]) successfully deleted
I20260807 03:02:25.582535 8019 raft_consensus.cc:2258] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:25.584431 8019 ts_tablet_manager.cc:1905] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:25.591606 8019 ts_tablet_manager.cc:1918] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:25.591770 8019 log.cc:1196] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/827efe3b93564266adaf4d2e84fa20f8
I20260807 03:02:25.592314 8019 ts_tablet_manager.cc:1939] T 827efe3b93564266adaf4d2e84fa20f8 P 858bf8708057484f84ee1e61946b73df: Deleting consensus metadata
I20260807 03:02:25.593978 7190 catalog_manager.cc:4841] TS 858bf8708057484f84ee1e61946b73df (127.5.122.2:38349): tablet 827efe3b93564266adaf4d2e84fa20f8 (table test-table [id=0a07c6e9df7c41a7940b088aa017c0fa]) successfully deleted
I20260807 03:02:25.610502 7993 raft_consensus.cc:482] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:25.610704 7993 raft_consensus.cc:504] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.612620 7993 leader_election.cc:288] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1:43605), 858bf8708057484f84ee1e61946b73df (127.5.122.2:38349)
I20260807 03:02:25.620841 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.612842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43605 (local address 127.5.122.3:38055)
0807 03:02:25.613646 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.613667 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.613824 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.614376 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.614382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.614411 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.614612 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.614624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.616120 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.616126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.617863 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.617884 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.619895 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.619901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.620204 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.620240 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.620352 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.620490 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682}
I20260807 03:02:25.621073 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.612943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:38055 (local address 127.5.122.1:43605)
0807 03:02:25.613147 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.613153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.613753 (+ 600us) server_negotiation.cc:407] Connection header received
0807 03:02:25.613982 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.613988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.614080 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.614176 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.614773 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.615976 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.618089 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.619736 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.619869 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.620021 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.620385 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.620520 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:02:25.622092 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.613314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:38349 (local address 127.5.122.3:42441)
0807 03:02:25.614080 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.614093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.614219 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.614535 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.614540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.614566 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.615256 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.615273 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.617031 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.617036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.619182 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.619195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.621078 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.621086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.621280 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.621313 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.621394 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.621514 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":598,"threads_started":1}
I20260807 03:02:25.622077 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.613374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:42441 (local address 127.5.122.2:38349)
0807 03:02:25.613545 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.613551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.614175 (+ 624us) server_negotiation.cc:407] Connection header received
0807 03:02:25.614364 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.614369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.614428 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.614498 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.615433 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.616883 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.619347 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.620636 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.620777 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.620886 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.621442 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.621566 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:02:25.621984 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d61f4f4d8e427bb51f8af3b3aaceed" candidate_uuid: "5042ba7deee74f28abc2984c8970d487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a94e3d559d4db1b35a1d62c37bc815" is_pre_election: true
I20260807 03:02:25.622262 7449 raft_consensus.cc:2454] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5042ba7deee74f28abc2984c8970d487 in term 0.
I20260807 03:02:25.622880 7789 leader_election.cc:302] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a94e3d559d4db1b35a1d62c37bc815, 5042ba7deee74f28abc2984c8970d487; no voters:
I20260807 03:02:25.623350 7993 raft_consensus.cc:2781] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:25.623513 7993 raft_consensus.cc:482] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:25.623629 7993 raft_consensus.cc:3037] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.623548 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d61f4f4d8e427bb51f8af3b3aaceed" candidate_uuid: "5042ba7deee74f28abc2984c8970d487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858bf8708057484f84ee1e61946b73df" is_pre_election: true
I20260807 03:02:25.624117 7683 raft_consensus.cc:2454] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5042ba7deee74f28abc2984c8970d487 in term 0.
I20260807 03:02:25.626448 7993 raft_consensus.cc:504] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.627065 7993 leader_election.cc:288] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [CANDIDATE]: Term 1 election: Requested vote from peers 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1:43605), 858bf8708057484f84ee1e61946b73df (127.5.122.2:38349)
I20260807 03:02:25.627574 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d61f4f4d8e427bb51f8af3b3aaceed" candidate_uuid: "5042ba7deee74f28abc2984c8970d487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858bf8708057484f84ee1e61946b73df"
I20260807 03:02:25.627758 7683 raft_consensus.cc:3037] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.627848 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d61f4f4d8e427bb51f8af3b3aaceed" candidate_uuid: "5042ba7deee74f28abc2984c8970d487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a94e3d559d4db1b35a1d62c37bc815"
I20260807 03:02:25.628085 7449 raft_consensus.cc:3037] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.631062 7683 raft_consensus.cc:2454] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5042ba7deee74f28abc2984c8970d487 in term 1.
I20260807 03:02:25.631459 7789 leader_election.cc:302] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5042ba7deee74f28abc2984c8970d487, 858bf8708057484f84ee1e61946b73df; no voters:
I20260807 03:02:25.631742 7993 raft_consensus.cc:2781] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:25.632064 7449 raft_consensus.cc:2454] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5042ba7deee74f28abc2984c8970d487 in term 1.
I20260807 03:02:25.632688 7993 raft_consensus.cc:686] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 1 LEADER]: Becoming Leader. State: Replica: 5042ba7deee74f28abc2984c8970d487, State: Running, Role: LEADER
I20260807 03:02:25.633244 7993 consensus_queue.cc:237] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.637729 7210 catalog_manager.cc:5495] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5042ba7deee74f28abc2984c8970d487 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "5042ba7deee74f28abc2984c8970d487" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } health_report { overall_health: HEALTHY } } }
Aug 07 03:02:25 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071743, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:25.702975 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.685028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:46261 (local address 127.0.0.1:39914)
0807 03:02:25.685819 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.685837 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.686014 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.686421 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.686426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.687458 (+ 1032us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:25.687839 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.687855 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.689318 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.689325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.690866 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.690883 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.691877 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.691884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.692080 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.692866 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:25.692893 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:25.696039 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:25.699855 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:25.699869 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:25.699877 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:25.700453 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:25.700895 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:25.700900 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:25.700903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:25.701074 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:25.701709 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:25.701718 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:25.701919 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.702099 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.702462 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":413,"threads_started":1}
I20260807 03:02:25.703418 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.685028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.5.122.62:46261)
0807 03:02:25.685230 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.685236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.685921 (+ 685us) server_negotiation.cc:407] Connection header received
0807 03:02:25.686142 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.686148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.686228 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.686384 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:25.688037 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.689150 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.691020 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.691680 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.691811 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.696234 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:25.696249 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:25.696254 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:25.696296 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:25.699692 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.700576 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.700582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.700585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.700768 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:25.701193 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:25.701199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:25.701203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:25.701573 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:25.701732 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.702230 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.702625 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":61}
I20260807 03:02:25.709919 7210 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:39914:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 03:02:25.711414 7210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:25.720005 7422 tablet_service.cc:1467] Processing CreateTablet for tablet 6bfc44e8890b435a84c84d609b20dab9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b882346cfb1d40039d4713fd8741fa1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:02:25.720670 7422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bfc44e8890b435a84c84d609b20dab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.723026 7850 tablet_service.cc:1467] Processing CreateTablet for tablet 6bfc44e8890b435a84c84d609b20dab9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b882346cfb1d40039d4713fd8741fa1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:02:25.723680 7850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bfc44e8890b435a84c84d609b20dab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.727288 7639 tablet_service.cc:1467] Processing CreateTablet for tablet 6bfc44e8890b435a84c84d609b20dab9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b882346cfb1d40039d4713fd8741fa1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:02:25.728097 7639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bfc44e8890b435a84c84d609b20dab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:25.729416 7986 tablet_bootstrap.cc:492] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: Bootstrap starting.
I20260807 03:02:25.732831 7986 tablet_bootstrap.cc:654] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.736253 7985 tablet_bootstrap.cc:492] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: Bootstrap starting.
I20260807 03:02:25.738397 7986 tablet_bootstrap.cc:492] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: No bootstrap required, opened a new log
I20260807 03:02:25.738620 7986 ts_tablet_manager.cc:1397] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260807 03:02:25.739636 7985 tablet_bootstrap.cc:654] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.739739 7986 raft_consensus.cc:348] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.740010 7986 raft_consensus.cc:374] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.740134 7986 raft_consensus.cc:729] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a94e3d559d4db1b35a1d62c37bc815, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.740490 7986 consensus_queue.cc:260] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.741614 7984 tablet_bootstrap.cc:492] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: Bootstrap starting.
I20260807 03:02:25.741168 7991 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.741881 7991 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:25.743937 7986 ts_tablet_manager.cc:1428] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 03:02:25.746306 7985 tablet_bootstrap.cc:492] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: No bootstrap required, opened a new log
I20260807 03:02:25.746555 7985 ts_tablet_manager.cc:1397] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.008s
I20260807 03:02:25.747660 7984 tablet_bootstrap.cc:654] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:25.747670 7985 raft_consensus.cc:348] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.748119 7985 raft_consensus.cc:374] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.748291 7985 raft_consensus.cc:729] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5042ba7deee74f28abc2984c8970d487, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.748612 7985 consensus_queue.cc:260] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.749393 7985 ts_tablet_manager.cc:1428] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:25.749302 7993 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.749899 7993 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:25.752573 7984 tablet_bootstrap.cc:492] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: No bootstrap required, opened a new log
I20260807 03:02:25.752856 7984 ts_tablet_manager.cc:1397] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 03:02:25.754202 7984 raft_consensus.cc:348] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.754658 7984 raft_consensus.cc:374] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:25.754848 7984 raft_consensus.cc:729] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858bf8708057484f84ee1e61946b73df, State: Initialized, Role: FOLLOWER
I20260807 03:02:25.755303 7984 consensus_queue.cc:260] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.756433 7984 ts_tablet_manager.cc:1428] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:25.756148 7992 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.756721 7992 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:25.879498 7991 raft_consensus.cc:482] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:25.879717 7991 raft_consensus.cc:504] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.881521 7991 leader_election.cc:288] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 858bf8708057484f84ee1e61946b73df (127.5.122.2:38349), 5042ba7deee74f28abc2984c8970d487 (127.5.122.3:39815)
I20260807 03:02:25.889734 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.881815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:42821 (local address 127.5.122.2:38349)
0807 03:02:25.882021 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.882027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.883788 (+ 1761us) server_negotiation.cc:407] Connection header received
0807 03:02:25.884015 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.884020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.884189 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.884284 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.885094 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.886106 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.887930 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.888788 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.888919 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.889024 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.889323 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.889463 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 03:02:25.889784 8044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.882306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:38349 (local address 127.5.122.1:42821)
0807 03:02:25.883688 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.883709 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.883869 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.884325 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.884330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.884357 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.884883 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.884897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.886234 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.886239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.887756 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.887769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.889019 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.889025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.889162 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.889185 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.889267 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.889406 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"mutex_wait_us":29,"thread_start_us":808,"threads_started":1}
I20260807 03:02:25.890620 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bfc44e8890b435a84c84d609b20dab9" candidate_uuid: "27a94e3d559d4db1b35a1d62c37bc815" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858bf8708057484f84ee1e61946b73df" is_pre_election: true
I20260807 03:02:25.890883 7683 raft_consensus.cc:2454] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a94e3d559d4db1b35a1d62c37bc815 in term 0.
I20260807 03:02:25.891367 7346 leader_election.cc:302] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a94e3d559d4db1b35a1d62c37bc815, 858bf8708057484f84ee1e61946b73df; no voters:
I20260807 03:02:25.891965 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.882864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39815 (local address 127.5.122.1:53817)
0807 03:02:25.883917 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.883936 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.884073 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.884492 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.884497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.884523 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:25.885443 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.885459 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.887205 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.887212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.889334 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.889352 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.891259 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.891268 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.891442 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.891475 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.891575 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.891690 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":706,"threads_started":1}
I20260807 03:02:25.892221 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.882176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:53817 (local address 127.5.122.3:39815)
0807 03:02:25.882409 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.882415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.884027 (+ 1612us) server_negotiation.cc:407] Connection header received
0807 03:02:25.884227 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.884233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.884309 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.884386 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:25.885624 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.886970 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.889554 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.890830 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.890983 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.891091 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.891709 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.891899 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 03:02:25.892529 7991 raft_consensus.cc:2781] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:25.892676 7991 raft_consensus.cc:482] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:25.892800 7991 raft_consensus.cc:3037] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.894784 7888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bfc44e8890b435a84c84d609b20dab9" candidate_uuid: "27a94e3d559d4db1b35a1d62c37bc815" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5042ba7deee74f28abc2984c8970d487" is_pre_election: true
I20260807 03:02:25.895076 7888 raft_consensus.cc:2454] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a94e3d559d4db1b35a1d62c37bc815 in term 0.
I20260807 03:02:25.896276 7991 raft_consensus.cc:504] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.897051 7991 leader_election.cc:288] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [CANDIDATE]: Term 1 election: Requested vote from peers 858bf8708057484f84ee1e61946b73df (127.5.122.2:38349), 5042ba7deee74f28abc2984c8970d487 (127.5.122.3:39815)
I20260807 03:02:25.897483 7888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bfc44e8890b435a84c84d609b20dab9" candidate_uuid: "27a94e3d559d4db1b35a1d62c37bc815" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5042ba7deee74f28abc2984c8970d487"
I20260807 03:02:25.897528 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bfc44e8890b435a84c84d609b20dab9" candidate_uuid: "27a94e3d559d4db1b35a1d62c37bc815" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858bf8708057484f84ee1e61946b73df"
I20260807 03:02:25.897662 7888 raft_consensus.cc:3037] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.897811 7683 raft_consensus.cc:3037] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:25.899770 7888 raft_consensus.cc:2454] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a94e3d559d4db1b35a1d62c37bc815 in term 1.
I20260807 03:02:25.900151 7349 leader_election.cc:302] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a94e3d559d4db1b35a1d62c37bc815, 5042ba7deee74f28abc2984c8970d487; no voters:
I20260807 03:02:25.900414 7991 raft_consensus.cc:2781] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:25.900889 7991 raft_consensus.cc:686] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 LEADER]: Becoming Leader. State: Replica: 27a94e3d559d4db1b35a1d62c37bc815, State: Running, Role: LEADER
I20260807 03:02:25.901292 7991 consensus_queue.cc:237] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } }
I20260807 03:02:25.901443 7683 raft_consensus.cc:2454] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a94e3d559d4db1b35a1d62c37bc815 in term 1.
I20260807 03:02:25.903699 8046 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: TxnStatusTablet state changed. Reason: New leader 27a94e3d559d4db1b35a1d62c37bc815. Latest consensus state: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.904176 8046 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:02:25.905068 8048 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 03:02:25.905381 8048 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 03:02:25.905797 7208 catalog_manager.cc:5495] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1). New cstate: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20260807 03:02:25.932929 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.927459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.5.122.1:43605)
0807 03:02:25.927631 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.927637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.927850 (+ 213us) server_negotiation.cc:407] Connection header received
0807 03:02:25.928123 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.928127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.928213 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.928334 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:25.928687 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.929479 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.930798 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.931234 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.931351 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.931784 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:25.932301 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:25.932440 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.932549 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.932634 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:02:25.932955 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.927438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43605 (local address 127.0.0.1:54246)
0807 03:02:25.927765 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.927781 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:25.927929 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:25.928371 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:25.928376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:25.928388 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:25.928562 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.928571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.929612 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.929618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.930657 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:25.930670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.931388 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.931394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:25.931549 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.931602 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:25.932419 (+ 817us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:25.932440 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 03:02:25.932519 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 03:02:25.932668 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260807 03:02:25.937640 7888 raft_consensus.cc:1261] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Refusing update from remote peer 27a94e3d559d4db1b35a1d62c37bc815: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:25.938089 7683 raft_consensus.cc:1261] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Refusing update from remote peer 27a94e3d559d4db1b35a1d62c37bc815: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:25.938287 7991 consensus_queue.cc:1035] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:25.939019 7991 consensus_queue.cc:1035] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:25.946645 7992 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: TxnStatusTablet state changed. Reason: New leader 27a94e3d559d4db1b35a1d62c37bc815. Latest consensus state: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.947019 7992 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:25.951992 7991 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.952406 7991 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:02:25.952682 8004 mvcc.cc:205] Tried to move back new op lower bound from 7315749871353126912 to 7315749871216115712. Current Snapshot: MvccSnapshot[applied={T|T < 7315749871353126912 or (T in {7315749871353126912})}]
I20260807 03:02:25.955412 8002 mvcc.cc:205] Tried to move back new op lower bound from 7315749871353126912 to 7315749871216115712. Current Snapshot: MvccSnapshot[applied={T|T < 7315749871353126912}]
I20260807 03:02:25.960719 7992 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.961323 7992 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 858bf8708057484f84ee1e61946b73df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:25.962213 7993 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: TxnStatusTablet state changed. Reason: New leader 27a94e3d559d4db1b35a1d62c37bc815. Latest consensus state: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.962882 7993 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:25.967068 8046 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.967401 8046 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 27a94e3d559d4db1b35a1d62c37bc815: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:02:25.969810 7993 tablet_replica.cc:439] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27a94e3d559d4db1b35a1d62c37bc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 } } peers { permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 } } peers { permanent_uuid: "5042ba7deee74f28abc2984c8970d487" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39815 } } }
I20260807 03:02:25.970412 7993 tablet_replica.cc:442] T 6bfc44e8890b435a84c84d609b20dab9 P 5042ba7deee74f28abc2984c8970d487: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:25.981640 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.965831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.5.122.3:39815)
0807 03:02:25.966199 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.966205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:25.966228 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:02:25.966358 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:25.966364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:25.966479 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:25.966658 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:25.967930 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.969596 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.976864 (+ 7268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:25.978066 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:25.978218 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:25.979373 (+ 1155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:25.980194 (+ 821us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:25.980348 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:25.981063 (+ 715us) server_negotiation.cc:299] Negotiation successful
0807 03:02:25.981162 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260807 03:02:26.009483 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.997885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:50823 (local address 127.5.122.1:43605)
0807 03:02:25.998121 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:25.998128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:26.000512 (+ 2384us) server_negotiation.cc:407] Connection header received
0807 03:02:26.000660 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:26.000667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:26.000827 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:26.000949 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:26.002175 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.003857 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.006705 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.007555 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.007710 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.008315 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:26.008524 (+ 209us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:26.008618 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:26.008845 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 03:02:26.008940 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:02:26.009676 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:25.998406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43605 (local address 127.5.122.3:50823)
0807 03:02:25.999550 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:25.999572 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:26.000001 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:26.001205 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:26.001210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:26.001224 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:26.001885 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:26.001902 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.004000 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.004006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:26.006539 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:26.006558 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.007785 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.007794 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:26.008007 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.008107 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:26.008649 (+ 542us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:26.008662 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 03:02:26.008781 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 03:02:26.008945 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"thread_start_us":578,"threads_started":1}
I20260807 03:02:26.024233 7683 raft_consensus.cc:1261] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Refusing update from remote peer 5042ba7deee74f28abc2984c8970d487: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:26.025058 8026 consensus_queue.cc:1035] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "858bf8708057484f84ee1e61946b73df" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:26.029597 7449 raft_consensus.cc:1261] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Refusing update from remote peer 5042ba7deee74f28abc2984c8970d487: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:26.030869 8026 consensus_queue.cc:1035] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a94e3d559d4db1b35a1d62c37bc815" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:26.033996 8003 mvcc.cc:205] Tried to move back new op lower bound from 7315749871706165248 to 7315749870117777408. Current Snapshot: MvccSnapshot[applied={T|T < 7315749871706165248}]
I20260807 03:02:26.092276 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:26.070918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:50475 (local address 127.5.122.3:39815)
0807 03:02:26.071099 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:26.071105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:26.079419 (+ 8314us) server_negotiation.cc:407] Connection header received
0807 03:02:26.083020 (+ 3601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:26.083026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:26.083121 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:26.083186 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:26.084400 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.085777 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.089185 (+ 3408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.090232 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.090452 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.091217 (+ 765us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:26.091404 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:26.091486 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:26.091707 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 03:02:26.091796 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:02:26.093721 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:26.070418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39815 (local address 127.5.122.1:50475)
0807 03:02:26.079308 (+ 8890us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:26.079332 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:26.082842 (+ 3510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:26.083394 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:26.083398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:26.083423 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:26.084215 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:26.084231 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.085909 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.085915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:26.089019 (+ 3104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:26.089035 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.090639 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.090649 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:26.090889 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.090976 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:26.091573 (+ 597us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:26.091583 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 03:02:26.092497 (+ 914us) client_negotiation.cc:240] Negotiation successful
0807 03:02:26.092699 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8576,"thread_start_us":10382,"threads_started":1}
I20260807 03:02:26.163635 7208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:26.163997 7208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:26.168668 7208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53fa177cdcf743618c9037095ba6955e: Sending DeleteTablet for 3 replicas of tablet 53d61f4f4d8e427bb51f8af3b3aaceed
I20260807 03:02:26.170667 7422 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d61f4f4d8e427bb51f8af3b3aaceed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:26 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:54234
I20260807 03:02:26.171185 7850 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d61f4f4d8e427bb51f8af3b3aaceed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:26 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:43194
I20260807 03:02:26.175761 7639 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d61f4f4d8e427bb51f8af3b3aaceed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:26 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:32870
I20260807 03:02:26.200218 8074 tablet_replica.cc:330] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: stopping tablet replica
I20260807 03:02:26.200783 8074 raft_consensus.cc:2229] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:26.202903 8074 raft_consensus.cc:2258] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:26.204808 8071 tablet_replica.cc:330] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: stopping tablet replica
I20260807 03:02:26.205699 8071 raft_consensus.cc:2229] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:26.208184 8071 raft_consensus.cc:2258] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:26.210031 8071 ts_tablet_manager.cc:1905] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:26.211174 8074 ts_tablet_manager.cc:1905] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:26.219357 8070 tablet_replica.cc:330] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: stopping tablet replica
I20260807 03:02:26.219597 8070 raft_consensus.cc:2229] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:26.219928 8070 raft_consensus.cc:2258] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:26.221325 8070 ts_tablet_manager.cc:1905] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:26.228678 8071 ts_tablet_manager.cc:1918] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:26.229259 8070 ts_tablet_manager.cc:1918] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:26.229450 8070 log.cc:1196] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/53d61f4f4d8e427bb51f8af3b3aaceed
I20260807 03:02:26.230010 8070 ts_tablet_manager.cc:1939] T 53d61f4f4d8e427bb51f8af3b3aaceed P 27a94e3d559d4db1b35a1d62c37bc815: Deleting consensus metadata
I20260807 03:02:26.231367 7190 catalog_manager.cc:4841] TS 27a94e3d559d4db1b35a1d62c37bc815 (127.5.122.1:43605): tablet 53d61f4f4d8e427bb51f8af3b3aaceed (table test-table [id=221c83a1df4e461794b367d10c982965]) successfully deleted
I20260807 03:02:26.232262 8071 log.cc:1196] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/53d61f4f4d8e427bb51f8af3b3aaceed
I20260807 03:02:26.232939 8071 ts_tablet_manager.cc:1939] T 53d61f4f4d8e427bb51f8af3b3aaceed P 5042ba7deee74f28abc2984c8970d487: Deleting consensus metadata
I20260807 03:02:26.234272 7192 catalog_manager.cc:4841] TS 5042ba7deee74f28abc2984c8970d487 (127.5.122.3:39815): tablet 53d61f4f4d8e427bb51f8af3b3aaceed (table test-table [id=221c83a1df4e461794b367d10c982965]) successfully deleted
I20260807 03:02:26.235353 8074 ts_tablet_manager.cc:1918] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
Aug 07 03:02:26 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:26.242450 8074 log.cc:1196] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/53d61f4f4d8e427bb51f8af3b3aaceed
I20260807 03:02:26.243321 8074 ts_tablet_manager.cc:1939] T 53d61f4f4d8e427bb51f8af3b3aaceed P 858bf8708057484f84ee1e61946b73df: Deleting consensus metadata
I20260807 03:02:26.249791 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:26.218610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.5.122.1:43605)
0807 03:02:26.218880 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:26.218889 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:26.221040 (+ 2151us) server_negotiation.cc:407] Connection header received
0807 03:02:26.221658 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:26.221666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:26.221768 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:26.221852 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:26.228953 (+ 7101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.230079 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.232170 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.232863 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.233006 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.239135 (+ 6129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:26.239152 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:26.239171 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:26.239221 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:26.245141 (+ 5920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.246366 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.246377 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.246380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.246669 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.247251 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.247258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.247262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.247737 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:26.248119 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:26.248821 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 03:02:26.249084 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 03:02:26.252075 7190 catalog_manager.cc:4841] TS 858bf8708057484f84ee1e61946b73df (127.5.122.2:38349): tablet 53d61f4f4d8e427bb51f8af3b3aaceed (table test-table [id=221c83a1df4e461794b367d10c982965]) successfully deleted
W20260807 03:02:26.252233 7521 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54260
I20260807 03:02:26.284088 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:26.265249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.5.122.62:46261)
0807 03:02:26.265850 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:26.265856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:26.266521 (+ 665us) server_negotiation.cc:407] Connection header received
0807 03:02:26.266781 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:26.266787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:26.266888 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:26.266961 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:26.269674 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.271095 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.273162 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.273826 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.273919 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.277161 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:26.277167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:26.277172 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:26.277212 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:26.280912 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.281712 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.281718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.281721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.281875 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.282349 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.282355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.282358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.282692 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:26.282914 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:26.283497 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 03:02:26.283720 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":239,"threads_started":1}
W20260807 03:02:26.285975 7208 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39930
I20260807 03:02:26.289273 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7320
I20260807 03:02:26.310493 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:26.290694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54264 (local address 127.5.122.1:43605)
0807 03:02:26.290884 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:26.290891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:26.298782 (+ 7891us) server_negotiation.cc:407] Connection header received
0807 03:02:26.298952 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:26.298955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:26.299022 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:26.299118 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:26.300272 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.301065 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.302263 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.302799 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.302916 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.305242 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:26.305246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:26.305250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:26.305281 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:26.307964 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.308581 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.308586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.308590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.308711 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.309049 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.309055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.309059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.309352 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:26.309600 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:26.309890 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 03:02:26.310112 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 03:02:26.311087 7521 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54264
W20260807 03:02:26.311753 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7320: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:26.311940 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 7320
I20260807 03:02:26.344561 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7535
Aug 07 03:02:26 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:02:26.360752 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:26.345321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.5.122.2:38349)
0807 03:02:26.345474 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:26.345479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:26.345543 (+ 64us) server_negotiation.cc:407] Connection header received
0807 03:02:26.345752 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:26.345758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:26.345836 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:26.345936 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:26.347536 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.348251 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.350013 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.350501 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.350615 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.355218 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:26.355232 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:26.355247 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:26.355284 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:26.357965 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.358629 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.358637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.358640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.358781 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.359137 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.359140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.359142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.359492 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:26.359828 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:26.360069 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 03:02:26.360298 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 03:02:26.361366 7742 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32880
W20260807 03:02:26.361999 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7535: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:26.362146 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 7535
I20260807 03:02:26.390441 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7756
Aug 07 03:02:26 dist-test-slave-fgb3 krb5kdc[7147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:02:26.405053 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:26.391161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.5.122.3:39815)
0807 03:02:26.391371 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:26.391377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:26.391396 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:02:26.391554 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:26.391558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:26.391623 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:26.391704 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:26.392887 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.393637 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.394917 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:26.395357 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:26.395467 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:26.399420 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:26.399435 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:26.399450 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:26.399486 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:26.402485 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.403089 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.403096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.403102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.403238 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:26.403586 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:26.403590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:26.403592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:26.403950 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:26.404263 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:26.404502 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 03:02:26.404718 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 03:02:26.405681 7950 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43208
W20260807 03:02:26.406297 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7756: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:26.406488 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 7756
I20260807 03:02:26.428918 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7165
I20260807 03:02:26.429982 7269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:26.544378 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 7165
2026-08-07T03:02:26Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4725 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:26 dist-test-slave-fgb3 krb5kdc[8086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:26 dist-test-slave-fgb3 krb5kdc[8086](info): set up 2 sockets
Aug 07 03:02:26 dist-test-slave-fgb3 krb5kdc[8086](info): commencing operation
krb5kdc: starting...
W20260807 03:02:27.700563 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:27 dist-test-slave-fgb3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:27.883708 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33883
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:28.196969 8102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:28.197238 8102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:28.197371 8102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:28.204370 8102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:28.204468 8102 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 03:02:28.204535 8102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:28.204625 8102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:28.204687 8102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:28.204770 8102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:28.214501 8102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33883
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:28.215713 8102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:28.217059 8102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:28.228091 8110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:28.228123 8109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:28.228163 8112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:28.228992 8102 server_base.cc:1034] running on GCE node
I20260807 03:02:28.229645 8102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:28.230932 8102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:28.232141 8102 hybrid_clock.cc:648] HybridClock initialized: now 1786071748232083 us; error 30 us; skew 500 ppm
I20260807 03:02:28.232445 8102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:28 dist-test-slave-fgb3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071748, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:28.235356 8102 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:28.236658 8102 webserver.cc:460] Webserver started at http://127.5.122.62:44471/ using document root <none> and password file <none>
I20260807 03:02:28.237313 8102 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:28.237404 8102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:28.237696 8102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:28.240588 8102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "fc4109519b1d48bcbb3475e2cd648670"
format_stamp: "Formatted at 2026-08-07 03:02:28 on dist-test-slave-fgb3"
server_key: "fb5c007bab3a5de6f015e5e4395d19ae"
server_key_iv: "cb9d0c4bd448f7b9c232776d399a9548"
server_key_version: "encryptionkey@0"
I20260807 03:02:28.241308 8102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "fc4109519b1d48bcbb3475e2cd648670"
format_stamp: "Formatted at 2026-08-07 03:02:28 on dist-test-slave-fgb3"
server_key: "fb5c007bab3a5de6f015e5e4395d19ae"
server_key_iv: "cb9d0c4bd448f7b9c232776d399a9548"
server_key_version: "encryptionkey@0"
I20260807 03:02:28.246094 8102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 03:02:28.249255 8126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:28.250514 8102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:02:28.250658 8102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "fc4109519b1d48bcbb3475e2cd648670"
format_stamp: "Formatted at 2026-08-07 03:02:28 on dist-test-slave-fgb3"
server_key: "fb5c007bab3a5de6f015e5e4395d19ae"
server_key_iv: "cb9d0c4bd448f7b9c232776d399a9548"
server_key_version: "encryptionkey@0"
I20260807 03:02:28.250849 8102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:28.286613 8102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:28.290339 8102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:28.290570 8102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:28.309607 8102 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:33883
I20260807 03:02:28.309626 8209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:33883 every 8 connection(s)
I20260807 03:02:28.310865 8102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:28.314381 8211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:28.320472 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 8102
I20260807 03:02:28.320657 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:28.321269 5608 external_mini_cluster.cc:1428] Setting key d1762a51811077ccda3fcfce13773384
I20260807 03:02:28.324229 8211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Bootstrap starting.
I20260807 03:02:28.327445 8211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:28.328541 8211 log.cc:824] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:28.331297 8211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: No bootstrap required, opened a new log
Aug 07 03:02:28 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:28.340142 8211 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4109519b1d48bcbb3475e2cd648670" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33883 } }
I20260807 03:02:28.340571 8211 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:28.340682 8211 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4109519b1d48bcbb3475e2cd648670, State: Initialized, Role: FOLLOWER
I20260807 03:02:28.341332 8211 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4109519b1d48bcbb3475e2cd648670" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33883 } }
I20260807 03:02:28.341616 8211 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:28.341764 8211 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:28.341984 8211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:28.343459 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:28.323083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.5.122.62:33883)
0807 03:02:28.323786 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:28.323797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:28.323835 (+ 38us) server_negotiation.cc:407] Connection header received
0807 03:02:28.325565 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:28.325589 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:28.325937 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:28.326217 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:28.327986 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.329331 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.330626 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.331601 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.331785 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:28.335358 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:28.335368 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:28.335386 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:28.335439 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:28.339207 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:28.339836 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:28.339844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:28.339852 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:28.340045 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:28.340359 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:28.340364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:28.340368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:28.341058 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:28.341333 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:28.341697 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 03:02:28.342113 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":333,"threads_started":1}
I20260807 03:02:28.345101 8211 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4109519b1d48bcbb3475e2cd648670" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33883 } }
I20260807 03:02:28.345558 8211 leader_election.cc:302] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc4109519b1d48bcbb3475e2cd648670; no voters:
I20260807 03:02:28.346117 8211 leader_election.cc:288] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:28.346210 8217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:28.346948 8217 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [term 1 LEADER]: Becoming Leader. State: Replica: fc4109519b1d48bcbb3475e2cd648670, State: Running, Role: LEADER
I20260807 03:02:28.347499 8217 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4109519b1d48bcbb3475e2cd648670" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33883 } }
I20260807 03:02:28.347991 8211 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:28.349905 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc4109519b1d48bcbb3475e2cd648670. Latest consensus state: current_term: 1 leader_uuid: "fc4109519b1d48bcbb3475e2cd648670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4109519b1d48bcbb3475e2cd648670" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33883 } } }
I20260807 03:02:28.350245 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:28.350579 8218 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc4109519b1d48bcbb3475e2cd648670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4109519b1d48bcbb3475e2cd648670" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33883 } } }
I20260807 03:02:28.351529 8218 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:28.352869 8226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:28.357826 8226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:28.366024 8226 catalog_manager.cc:1347] Generated new cluster ID: 781ebb9264cc46c1be3bb427e194bae8
I20260807 03:02:28.366122 8226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:28.389362 8226 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:28.390647 8226 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:28.401556 8226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Generated new TSK 0
I20260807 03:02:28.402282 8226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:02:28.458583 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33883
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:28.782281 8240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:28.782596 8240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:28.782712 8240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:28.789656 8240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:28.789755 8240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:28.789852 8240 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 03:02:28.789904 8240 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:02:28.789973 8240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:28.799561 8240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:33883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:28.800776 8240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:28.802049 8240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:28.814491 8251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:28.814468 8249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:28.814708 8247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:28.815281 8240 server_base.cc:1034] running on GCE node
I20260807 03:02:28.815827 8240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:28.816645 8240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:28.817842 8240 hybrid_clock.cc:648] HybridClock initialized: now 1786071748817814 us; error 49 us; skew 500 ppm
I20260807 03:02:28.818434 8240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:28 dist-test-slave-fgb3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071748, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:28.821381 8240 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:28.822765 8240 webserver.cc:460] Webserver started at http://127.5.122.1:34375/ using document root <none> and password file <none>
I20260807 03:02:28.823400 8240 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:28.823515 8240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:28.823809 8240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:28.826622 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "064a4f51415d47ab9bc5fdf39943172a"
format_stamp: "Formatted at 2026-08-07 03:02:28 on dist-test-slave-fgb3"
server_key: "4289c8feaa49bd1e4447273cfbf4335c"
server_key_iv: "79e992d9903d9386b24169f8ef6f7ebd"
server_key_version: "encryptionkey@0"
I20260807 03:02:28.827354 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "064a4f51415d47ab9bc5fdf39943172a"
format_stamp: "Formatted at 2026-08-07 03:02:28 on dist-test-slave-fgb3"
server_key: "4289c8feaa49bd1e4447273cfbf4335c"
server_key_iv: "79e992d9903d9386b24169f8ef6f7ebd"
server_key_version: "encryptionkey@0"
I20260807 03:02:28.832232 8240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:02:28.835632 8262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:28.836853 8240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:28.837004 8240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "064a4f51415d47ab9bc5fdf39943172a"
format_stamp: "Formatted at 2026-08-07 03:02:28 on dist-test-slave-fgb3"
server_key: "4289c8feaa49bd1e4447273cfbf4335c"
server_key_iv: "79e992d9903d9386b24169f8ef6f7ebd"
server_key_version: "encryptionkey@0"
I20260807 03:02:28.837170 8240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:28.852121 8240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:28.856082 8240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:28.856343 8240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:28.858901 8240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:28.859005 8240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:28.859131 8240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:28.859256 8240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 07 03:02:28 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071748, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:28.881800 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:28.859302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:38053 (local address 127.5.122.62:33883)
0807 03:02:28.859762 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:28.859768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:28.860876 (+ 1108us) server_negotiation.cc:407] Connection header received
0807 03:02:28.863032 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:28.863039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:28.863129 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:28.863245 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:28.865264 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.866326 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.869922 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.870577 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.870710 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:28.875123 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:28.875130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:28.875135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:28.875178 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:28.878164 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:28.879104 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:28.879110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:28.879114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:28.879265 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:28.879874 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:28.879879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:28.879882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:28.880203 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:28.880348 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:28.881359 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0807 03:02:28.881542 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":235,"threads_started":1}
I20260807 03:02:28.883580 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:28.859849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33883 (local address 127.5.122.1:38053)
0807 03:02:28.860704 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:28.860747 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:28.862763 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:28.863509 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:28.863517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:28.864185 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:28.865023 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:28.865052 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.866497 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.866502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:28.869760 (+ 3258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:28.869778 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.870833 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.870841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:28.871023 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:28.871775 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:28.871810 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:28.874856 (+ 3046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:28.878349 (+ 3493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:28.878362 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:28.878377 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:28.878980 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:28.879393 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:28.879398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:28.879406 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:28.879719 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:28.880326 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:28.880335 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:28.880855 (+ 520us) client_negotiation.cc:769] Sending connection context
0807 03:02:28.881244 (+ 389us) client_negotiation.cc:240] Negotiation successful
0807 03:02:28.881836 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":356,"threads_started":1}
I20260807 03:02:28.898411 8240 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:33515
I20260807 03:02:28.898432 8435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:33515 every 8 connection(s)
I20260807 03:02:28.899696 8240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:28.901971 8437 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33883
I20260807 03:02:28.902199 8437 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:28.903030 8437 heartbeater.cc:511] Master 127.5.122.62:33883 requested a full tablet report, sending...
I20260807 03:02:28.904706 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:28.890627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33883 (local address 127.5.122.1:39071)
0807 03:02:28.891325 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:28.891341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:28.891480 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:28.892112 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:28.892118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:28.892805 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:28.893316 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:28.893333 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.894580 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.894586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:28.896455 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:28.896473 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.897200 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.897207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:28.897354 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:28.898064 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:28.898083 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:28.899084 (+ 1001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:28.902167 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:28.902172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:28.902175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:28.902725 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:28.903085 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:28.903090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:28.903093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:28.903237 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:28.903877 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:28.903883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:28.904073 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 03:02:28.904194 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 03:02:28.904415 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":392,"threads_started":1}
I20260807 03:02:28.905033 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:28.890551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:39071 (local address 127.5.122.62:33883)
0807 03:02:28.890728 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:28.890733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:28.891418 (+ 685us) server_negotiation.cc:407] Connection header received
0807 03:02:28.891661 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:28.891667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:28.891750 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:28.891867 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:28.893481 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.894451 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.896596 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:28.897004 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:28.897115 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:28.899253 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:28.899258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:28.899261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:28.899291 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:28.902058 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:28.902835 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:28.902839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:28.902842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:28.902966 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:28.903375 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:28.903380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:28.903383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:28.903749 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:28.903899 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:28.904346 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 03:02:28.904578 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 03:02:28.905174 8148 ts_manager.cc:194] Registered new tserver with Master: 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515)
I20260807 03:02:28.906752 8148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:38053
I20260807 03:02:28.909070 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 8240
I20260807 03:02:28.909247 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:28.909862 5608 external_mini_cluster.cc:1428] Setting key 68a3e2d4806397346e6d0d16d1de1976
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:02:28.950762 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33883
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:29.265697 8442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:29.265988 8442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:29.266144 8442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:29.273196 8442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:29.273309 8442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:29.273443 8442 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 03:02:29.273514 8442 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:02:29.273619 8442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:29.283130 8442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:33883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:29.284315 8442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:29.285574 8442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:29.297727 8449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:29.297920 8453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:29.297955 8451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:29.299140 8442 server_base.cc:1034] running on GCE node
I20260807 03:02:29.299669 8442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:29.300469 8442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:29.301654 8442 hybrid_clock.cc:648] HybridClock initialized: now 1786071749301607 us; error 41 us; skew 500 ppm
I20260807 03:02:29.301911 8442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:29 dist-test-slave-fgb3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:29.304801 8442 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:29.306053 8442 webserver.cc:460] Webserver started at http://127.5.122.2:36263/ using document root <none> and password file <none>
I20260807 03:02:29.306721 8442 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:29.306856 8442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:29.307121 8442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:29.310170 8442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "831e70d9006f494ba1a6987f7287f9bc"
format_stamp: "Formatted at 2026-08-07 03:02:29 on dist-test-slave-fgb3"
server_key: "9b3a1b6fa6454466787972127e58e8ea"
server_key_iv: "6aaf878c6adff61c4161894f0cab8d82"
server_key_version: "encryptionkey@0"
I20260807 03:02:29.311160 8442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "831e70d9006f494ba1a6987f7287f9bc"
format_stamp: "Formatted at 2026-08-07 03:02:29 on dist-test-slave-fgb3"
server_key: "9b3a1b6fa6454466787972127e58e8ea"
server_key_iv: "6aaf878c6adff61c4161894f0cab8d82"
server_key_version: "encryptionkey@0"
I20260807 03:02:29.318133 8442 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 03:02:29.322293 8462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:29.323900 8442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:29.324066 8442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "831e70d9006f494ba1a6987f7287f9bc"
format_stamp: "Formatted at 2026-08-07 03:02:29 on dist-test-slave-fgb3"
server_key: "9b3a1b6fa6454466787972127e58e8ea"
server_key_iv: "6aaf878c6adff61c4161894f0cab8d82"
server_key_version: "encryptionkey@0"
I20260807 03:02:29.324273 8442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:29.350536 8442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:29.354862 8442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:29.355134 8442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:29.357815 8442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:29.357911 8442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:29.358032 8442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:29.358117 8442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:02:29 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:29.380821 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.358412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:60137 (local address 127.5.122.62:33883)
0807 03:02:29.358609 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.358614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.360100 (+ 1486us) server_negotiation.cc:407] Connection header received
0807 03:02:29.362551 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.362558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.362687 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.362816 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:29.365002 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.365986 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.368397 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.368970 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.369104 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.373794 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:29.373801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:29.373806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:29.373849 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:29.376910 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.377962 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.377968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.377972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.378130 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.378714 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.378718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.378720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.379100 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:29.379253 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.380237 (+ 984us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.380484 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 03:02:29.381824 8476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.359050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33883 (local address 127.5.122.2:60137)
0807 03:02:29.359911 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:29.359965 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:29.362172 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:29.363107 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:29.363122 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:29.363910 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:29.364762 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.364805 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.366147 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.366151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.368242 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.368257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.369269 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.369278 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.369597 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.370412 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:29.370459 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:29.373573 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:29.377079 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.377086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.377097 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.377695 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.378246 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.378251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.378255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.378571 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.379318 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:29.379324 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:29.379805 (+ 481us) client_negotiation.cc:769] Sending connection context
0807 03:02:29.380100 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 03:02:29.380637 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":380,"threads_started":1}
I20260807 03:02:29.398785 8442 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:34711
I20260807 03:02:29.398792 8633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:34711 every 8 connection(s)
I20260807 03:02:29.400324 8442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:29.402714 8634 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33883
I20260807 03:02:29.402943 8634 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:29.403470 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.388768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:60793 (local address 127.5.122.62:33883)
0807 03:02:29.388911 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.388915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.389569 (+ 654us) server_negotiation.cc:407] Connection header received
0807 03:02:29.389724 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.389729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.389803 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.389904 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:29.391146 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.392111 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.393951 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.394471 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.394589 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.396999 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:29.397005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:29.397009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:29.397045 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:29.400355 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.401175 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.401180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.401183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.401337 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.401803 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.401806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.401808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.402306 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:29.402471 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.402925 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.403129 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 03:02:29.403584 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.388880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33883 (local address 127.5.122.2:60793)
0807 03:02:29.389471 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:29.389486 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:29.389616 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:29.389947 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:29.389951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:29.390613 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:29.391012 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.391027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.392243 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.392248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.393819 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.393835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.394837 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.394856 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.395048 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.395664 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:29.395687 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:29.396798 (+ 1111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:29.400521 (+ 3723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.400525 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.400528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.401019 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.401478 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.401482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.401486 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.401654 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.402438 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:29.402443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:29.402655 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 03:02:29.402823 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 03:02:29.403138 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":326,"threads_started":1}
I20260807 03:02:29.403651 8634 heartbeater.cc:511] Master 127.5.122.62:33883 requested a full tablet report, sending...
I20260807 03:02:29.405037 8148 ts_manager.cc:194] Registered new tserver with Master: 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711)
I20260807 03:02:29.405788 8148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:60137
I20260807 03:02:29.408636 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 8442
I20260807 03:02:29.408815 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:29.409227 5608 external_mini_cluster.cc:1428] Setting key b11031458c6f6e4c525358385472c2c0
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:29.447549 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33883
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:29.759326 8640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:29.759591 8640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:29.759692 8640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:29.766744 8640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:29.766840 8640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:29.766923 8640 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 03:02:29.766973 8640 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:02:29.767040 8640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:29.776319 8640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:33883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:29.777477 8640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:29.778774 8640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:29.791553 8645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:29.791525 8647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:29.791647 8649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:29.792229 8640 server_base.cc:1034] running on GCE node
I20260807 03:02:29.792817 8640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:29.793672 8640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:29.794862 8640 hybrid_clock.cc:648] HybridClock initialized: now 1786071749794807 us; error 47 us; skew 500 ppm
I20260807 03:02:29.795157 8640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:29 dist-test-slave-fgb3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:29.802889 8640 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:29.804282 8640 webserver.cc:460] Webserver started at http://127.5.122.3:36075/ using document root <none> and password file <none>
I20260807 03:02:29.804929 8640 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:29.805045 8640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:29.805336 8640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:29.808173 8640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
format_stamp: "Formatted at 2026-08-07 03:02:29 on dist-test-slave-fgb3"
server_key: "d6603b127b64e24462cc7430ecc22c79"
server_key_iv: "3e79dacfa71eb9dbb0eee7c1222cf07b"
server_key_version: "encryptionkey@0"
I20260807 03:02:29.808936 8640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
format_stamp: "Formatted at 2026-08-07 03:02:29 on dist-test-slave-fgb3"
server_key: "d6603b127b64e24462cc7430ecc22c79"
server_key_iv: "3e79dacfa71eb9dbb0eee7c1222cf07b"
server_key_version: "encryptionkey@0"
I20260807 03:02:29.813817 8640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:02:29.817014 8661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:29.818204 8640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 03:02:29.818385 8640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
format_stamp: "Formatted at 2026-08-07 03:02:29 on dist-test-slave-fgb3"
server_key: "d6603b127b64e24462cc7430ecc22c79"
server_key_iv: "3e79dacfa71eb9dbb0eee7c1222cf07b"
server_key_version: "encryptionkey@0"
I20260807 03:02:29.818557 8640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:29.835058 8640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:29.838625 8640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:29.838871 8640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:29.841492 8640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:29.841595 8640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:29.841714 8640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:29.841833 8640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 03:02:29 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:29.861658 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.841890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:47919 (local address 127.5.122.62:33883)
0807 03:02:29.842101 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.842108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.843448 (+ 1340us) server_negotiation.cc:407] Connection header received
0807 03:02:29.845385 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.845391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.845493 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.845615 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:29.847671 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.848862 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.850611 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.851182 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.851316 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.855835 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:29.855841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:29.855844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:29.855877 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:29.858300 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.859270 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.859275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.859278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.859410 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.859989 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.859994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.859998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.860248 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:29.860366 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.861226 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.861407 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:02:29.863384 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.842371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33883 (local address 127.5.122.3:47919)
0807 03:02:29.843266 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:29.843322 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:29.845114 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:29.845889 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:29.845904 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:29.846715 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:29.847481 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.847508 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.849038 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.849044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.850476 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.850489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.851441 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.851450 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.851764 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.852520 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:29.852551 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:29.855628 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:29.858463 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.858471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.858486 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.859102 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.859533 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.859539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.859543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.859835 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.860359 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:29.860368 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:29.860811 (+ 443us) client_negotiation.cc:769] Sending connection context
0807 03:02:29.861122 (+ 311us) client_negotiation.cc:240] Negotiation successful
0807 03:02:29.861663 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":395,"threads_started":1}
I20260807 03:02:29.882472 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.869061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:37999 (local address 127.5.122.62:33883)
0807 03:02:29.869229 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.869233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.869969 (+ 736us) server_negotiation.cc:407] Connection header received
0807 03:02:29.870114 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.870120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.870183 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.870260 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:29.871524 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.872616 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.873892 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.874507 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.874631 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.877142 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:29.877149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:29.877153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:29.877191 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:29.880008 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.880606 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.880612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.880615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.880736 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.881079 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.881084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.881088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.881376 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:29.881505 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.881917 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.882116 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:02:29.883517 8742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.869163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33883 (local address 127.5.122.3:37999)
0807 03:02:29.869877 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:29.869894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:29.870011 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:29.870296 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:29.870300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:29.870869 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:29.871290 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.871305 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.872755 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.872760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.873764 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.873776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.874888 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.874895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.875159 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.875826 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:29.875851 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:29.876900 (+ 1049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:29.880125 (+ 3225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.880130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.880134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.880500 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.880837 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:29.880842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:29.880845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:29.880992 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:29.881485 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:29.881489 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:29.881661 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 03:02:29.881780 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 03:02:29.882095 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":405,"threads_started":1}
I20260807 03:02:29.893783 8640 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:44445
I20260807 03:02:29.893831 8822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:44445 every 8 connection(s)
I20260807 03:02:29.895007 8640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:29.897112 8823 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33883
I20260807 03:02:29.897338 8823 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:29.898046 8823 heartbeater.cc:511] Master 127.5.122.62:33883 requested a full tablet report, sending...
I20260807 03:02:29.899466 8148 ts_manager.cc:194] Registered new tserver with Master: 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:29.900352 8148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:47919
I20260807 03:02:29.903781 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 8640
I20260807 03:02:29.903939 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:29.904330 5608 external_mini_cluster.cc:1428] Setting key fc4a1138514ec86e48e65e1ac6e80653
I20260807 03:02:29.907039 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:02:29.909478 8437 heartbeater.cc:503] Master 127.5.122.62:33883 was elected leader, sending a full tablet report...
Aug 07 03:02:29 dist-test-slave-fgb3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 03:02:29 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:29.953948 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.940947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33560 (local address 127.5.122.62:33883)
0807 03:02:29.941174 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.941180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.941378 (+ 198us) server_negotiation.cc:407] Connection header received
0807 03:02:29.941575 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.941597 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.941663 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.941736 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:29.943096 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.943811 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.944999 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.945406 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.945491 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.949127 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:29.949133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:29.949136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:29.949170 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:29.951650 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.952221 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.952226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.952229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.952374 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:29.952660 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:29.952663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:29.952668 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:29.952970 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:29.953172 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.953397 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.953587 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 03:02:29.957731 8148 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:29.964534 8148 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:29.987529 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.976094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:33515 (local address 127.0.0.1:59440)
0807 03:02:29.976936 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:29.976980 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:29.977140 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:29.978230 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:29.978252 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:29.978292 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:29.978804 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.978821 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.981137 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.981144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.983770 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.983790 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.986342 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.986351 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.986604 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.986675 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 03:02:29.986817 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 03:02:29.987031 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":421,"threads_started":1}
I20260807 03:02:29.988682 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.976404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.5.122.1:33515)
0807 03:02:29.977264 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.977276 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.977308 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:02:29.977413 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.977422 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.977690 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.977909 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:29.979061 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.980828 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.984052 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.985840 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.985984 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.988011 (+ 2027us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.988101 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.988311 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":751,"threads_started":1}
I20260807 03:02:29.990301 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.977345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:34711 (local address 127.0.0.1:41628)
0807 03:02:29.978394 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:29.978415 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:29.978552 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:29.980257 (+ 1705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:29.980263 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:29.980290 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:29.980751 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.980762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.983945 (+ 3183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.983951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.986153 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.986169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.989467 (+ 3298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.989475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.989671 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.989706 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 03:02:29.989800 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 03:02:29.989946 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"thread_start_us":781,"threads_started":1}
I20260807 03:02:29.990605 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.977597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.5.122.2:34711)
0807 03:02:29.978092 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.978100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.978689 (+ 589us) server_negotiation.cc:407] Connection header received
0807 03:02:29.978795 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.978802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.979034 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.980977 (+ 1943us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:29.981195 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.983769 (+ 2574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.986383 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.988955 (+ 2572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.989081 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.989289 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.990081 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.990274 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":214,"threads_started":1}
I20260807 03:02:29.993638 8332 tablet_service.cc:1467] Processing CreateTablet for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c (DEFAULT_TABLE table=test-table [id=8108e7e65e3e4364b9806b9acb38d5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:29.994345 8534 tablet_service.cc:1467] Processing CreateTablet for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c (DEFAULT_TABLE table=test-table [id=8108e7e65e3e4364b9806b9acb38d5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:29.995043 8332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:29.995479 8534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:29.998952 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.978537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:44445 (local address 127.0.0.1:59110)
0807 03:02:29.989135 (+ 10598us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:29.989158 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:29.989280 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:29.991827 (+ 2547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:29.991832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:29.991862 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:29.992029 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.992041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.994644 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.994649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.996453 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:29.996470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.998224 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.998229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:29.998465 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.998499 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 03:02:29.998565 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 03:02:29.998692 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10470,"thread_start_us":11280,"threads_started":1}
I20260807 03:02:29.999758 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:29.978869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.5.122.3:44445)
0807 03:02:29.991235 (+ 12366us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:29.991242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:29.991274 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:02:29.991379 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:29.991386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:29.991612 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:29.992302 (+ 690us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:29.992901 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.994490 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.996639 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:29.998043 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:29.998887 (+ 844us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:29.999138 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:29.999213 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 03:02:29.999398 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12204,"thread_start_us":12046,"threads_started":1}
I20260807 03:02:30.003489 8737 tablet_service.cc:1467] Processing CreateTablet for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c (DEFAULT_TABLE table=test-table [id=8108e7e65e3e4364b9806b9acb38d5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:30.004940 8737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:30.007881 8847 tablet_bootstrap.cc:492] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Bootstrap starting.
I20260807 03:02:30.009162 8848 tablet_bootstrap.cc:492] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Bootstrap starting.
I20260807 03:02:30.011830 8847 tablet_bootstrap.cc:654] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.012727 8848 tablet_bootstrap.cc:654] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.013199 8847 log.cc:824] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:30.013980 8848 log.cc:824] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:30.017869 8851 tablet_bootstrap.cc:492] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Bootstrap starting.
I20260807 03:02:30.020696 8848 tablet_bootstrap.cc:492] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: No bootstrap required, opened a new log
I20260807 03:02:30.020974 8848 ts_tablet_manager.cc:1397] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260807 03:02:30.021114 8847 tablet_bootstrap.cc:492] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: No bootstrap required, opened a new log
I20260807 03:02:30.021401 8847 ts_tablet_manager.cc:1397] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260807 03:02:30.021507 8851 tablet_bootstrap.cc:654] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.022794 8851 log.cc:824] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:30.025655 8851 tablet_bootstrap.cc:492] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: No bootstrap required, opened a new log
I20260807 03:02:30.025951 8851 ts_tablet_manager.cc:1397] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 03:02:30.029673 8848 raft_consensus.cc:348] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.030058 8848 raft_consensus.cc:374] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.030166 8848 raft_consensus.cc:729] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.030781 8848 consensus_queue.cc:260] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.031214 8847 raft_consensus.cc:348] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.031672 8847 raft_consensus.cc:374] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.031785 8847 raft_consensus.cc:729] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064a4f51415d47ab9bc5fdf39943172a, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.031840 8634 heartbeater.cc:503] Master 127.5.122.62:33883 was elected leader, sending a full tablet report...
I20260807 03:02:30.032478 8848 ts_tablet_manager.cc:1428] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260807 03:02:30.032370 8847 consensus_queue.cc:260] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.034075 8847 ts_tablet_manager.cc:1428] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
I20260807 03:02:30.036954 8851 raft_consensus.cc:348] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.037465 8851 raft_consensus.cc:374] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.037581 8851 raft_consensus.cc:729] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163f3e6f66504b3e8f5b02dfdf3bdf48, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.038360 8851 consensus_queue.cc:260] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.039561 8823 heartbeater.cc:503] Master 127.5.122.62:33883 was elected leader, sending a full tablet report...
I20260807 03:02:30.040163 8851 ts_tablet_manager.cc:1428] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260807 03:02:30.079105 8853 raft_consensus.cc:482] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:30.079422 8853 raft_consensus.cc:504] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.080948 8856 raft_consensus.cc:482] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:30.081099 8853 leader_election.cc:288] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:30.081362 8856 raft_consensus.cc:504] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.086028 8856 leader_election.cc:288] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:30.089972 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.082518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:59489 (local address 127.5.122.3:44445)
0807 03:02:30.082697 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.082703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.083180 (+ 477us) server_negotiation.cc:407] Connection header received
0807 03:02:30.083393 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.083398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.083478 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.083555 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:30.084263 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.085272 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.087803 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.089010 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.089117 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.089221 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.089573 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.089705 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":43}
I20260807 03:02:30.090773 8861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.082170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:44445 (local address 127.5.122.2:59489)
0807 03:02:30.083049 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.083067 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.083233 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.083575 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.083579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.083600 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:30.084056 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.084069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.085458 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.085463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.087643 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.087659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.089202 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.089208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.089381 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.089415 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.090142 (+ 727us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.090282 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":1,"thread_start_us":549,"threads_started":1}
I20260807 03:02:30.091032 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.081597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:34927 (local address 127.5.122.1:33515)
0807 03:02:30.081811 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.081817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.082190 (+ 373us) server_negotiation.cc:407] Connection header received
0807 03:02:30.082553 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.082560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.082659 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.082731 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:30.083495 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.084701 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.088394 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.089650 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.089761 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.089854 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.090632 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.090791 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":73}
I20260807 03:02:30.091224 8860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.081477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:33515 (local address 127.5.122.2:34927)
0807 03:02:30.082094 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.082113 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.082360 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.082952 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.082956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.082987 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:30.083337 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.083359 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.084866 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.084871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.088192 (+ 3321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.088207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.089995 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.090001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.090437 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.090475 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.090679 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.090820 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":1112,"thread_start_us":298,"threads_started":1}
I20260807 03:02:30.092414 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.083961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:51137 (local address 127.5.122.2:34711)
0807 03:02:30.084145 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.084150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.084771 (+ 621us) server_negotiation.cc:407] Connection header received
0807 03:02:30.085034 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.085039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.085105 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.085156 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:30.085962 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.087114 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.089750 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.090986 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.091093 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.091194 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.092065 (+ 871us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.092189 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:02:30.092196 8767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" is_pre_election: true
I20260807 03:02:30.092398 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a" is_pre_election: true
I20260807 03:02:30.092782 8767 raft_consensus.cc:2454] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 0.
I20260807 03:02:30.092828 8363 raft_consensus.cc:2454] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 0.
I20260807 03:02:30.093287 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.083713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:34711 (local address 127.5.122.1:51137)
0807 03:02:30.084682 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.084700 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.084875 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.085351 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.085355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.085381 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:30.085781 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.085793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.087291 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.087296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.089570 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.089585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.091645 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.091650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.091886 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.091920 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.092768 (+ 848us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.092912 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":40,"thread_start_us":647,"threads_started":1}
I20260807 03:02:30.093531 8465 leader_election.cc:302] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163f3e6f66504b3e8f5b02dfdf3bdf48, 831e70d9006f494ba1a6987f7287f9bc; no voters:
I20260807 03:02:30.094089 8853 raft_consensus.cc:2781] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:30.094235 8853 raft_consensus.cc:482] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:30.094379 8853 raft_consensus.cc:3037] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:30.094290 8567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831e70d9006f494ba1a6987f7287f9bc" is_pre_election: true
I20260807 03:02:30.095305 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.084866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:44445 (local address 127.5.122.1:49465)
0807 03:02:30.087133 (+ 2267us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.087154 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.087268 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.087804 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.087809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.087832 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:30.088257 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.088270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.090255 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.090260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.092865 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.092879 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.094635 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.094642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.094803 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.094835 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.094923 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.095043 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2138,"thread_start_us":1938,"threads_started":1}
I20260807 03:02:30.095417 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.085193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:49465 (local address 127.5.122.3:44445)
0807 03:02:30.086669 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.086675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.087360 (+ 685us) server_negotiation.cc:407] Connection header received
0807 03:02:30.087416 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.087421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.087484 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.087567 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:30.088480 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.090100 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.093043 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.094219 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.094372 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.094485 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.094980 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.095111 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1343,"thread_start_us":1155,"threads_started":1}
I20260807 03:02:30.095990 8767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" is_pre_election: true
I20260807 03:02:30.096195 8767 raft_consensus.cc:2454] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 064a4f51415d47ab9bc5fdf39943172a in term 0.
I20260807 03:02:30.096588 8264 leader_election.cc:302] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064a4f51415d47ab9bc5fdf39943172a, 163f3e6f66504b3e8f5b02dfdf3bdf48; no voters:
I20260807 03:02:30.096925 8856 raft_consensus.cc:2781] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:30.097041 8856 raft_consensus.cc:482] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:30.097133 8856 raft_consensus.cc:3037] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:30.097117 8853 raft_consensus.cc:504] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.097671 8567 raft_consensus.cc:2379] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 064a4f51415d47ab9bc5fdf39943172a in current term 1: Already voted for candidate 831e70d9006f494ba1a6987f7287f9bc in this term.
I20260807 03:02:30.097854 8853 leader_election.cc:288] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Requested vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:30.098294 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a"
I20260807 03:02:30.098686 8767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
I20260807 03:02:30.098889 8767 raft_consensus.cc:3037] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:30.100279 8856 raft_consensus.cc:504] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.100634 8363 raft_consensus.cc:2379] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 831e70d9006f494ba1a6987f7287f9bc in current term 1: Already voted for candidate 064a4f51415d47ab9bc5fdf39943172a in this term.
I20260807 03:02:30.100926 8856 leader_election.cc:288] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 election: Requested vote from peers 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:30.101378 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
I20260807 03:02:30.101743 8767 raft_consensus.cc:2454] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 1.
I20260807 03:02:30.102164 8465 leader_election.cc:302] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 163f3e6f66504b3e8f5b02dfdf3bdf48, 831e70d9006f494ba1a6987f7287f9bc; no voters: 064a4f51415d47ab9bc5fdf39943172a
I20260807 03:02:30.102376 8567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0a2cfed3d94ebf8c8a34c5e774675c" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831e70d9006f494ba1a6987f7287f9bc"
I20260807 03:02:30.102466 8853 raft_consensus.cc:2781] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:30.102900 8853 raft_consensus.cc:686] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 1 LEADER]: Becoming Leader. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Running, Role: LEADER
I20260807 03:02:30.103315 8853 consensus_queue.cc:237] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.105844 8264 leader_election.cc:302] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 064a4f51415d47ab9bc5fdf39943172a; no voters: 163f3e6f66504b3e8f5b02dfdf3bdf48, 831e70d9006f494ba1a6987f7287f9bc
I20260807 03:02:30.106115 8856 raft_consensus.cc:2726] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 03:02:30.107627 8148 catalog_manager.cc:5495] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2). New cstate: current_term: 1 leader_uuid: "831e70d9006f494ba1a6987f7287f9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } health_report { overall_health: UNKNOWN } } }
W20260807 03:02:30.146548 8824 tablet.cc:2369] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:02:30.151966 8438 tablet.cc:2369] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:02:30.152727 8635 tablet.cc:2369] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 03:02:30 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071748, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:30.169013 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.156641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33883 (local address 127.0.0.1:33564)
0807 03:02:30.157232 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.157248 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.157381 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.157651 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.157654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.158354 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:30.158648 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.158657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.159606 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.159609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.160568 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.160579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.161277 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.161281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.161409 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.161961 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:30.161977 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:30.164129 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:30.166772 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:30.166782 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:30.166788 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:30.167210 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:30.167527 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:30.167531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:30.167533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:30.167673 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:30.168154 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:30.168161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:30.168306 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.168456 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.168661 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":345,"threads_started":1}
I20260807 03:02:30.169041 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.156573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.5.122.62:33883)
0807 03:02:30.156757 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.156763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.157325 (+ 562us) server_negotiation.cc:407] Connection header received
0807 03:02:30.157473 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.157477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.157539 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.157621 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:30.158784 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.159499 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.160682 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.161126 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.161213 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.164277 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:30.164282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:30.164285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:30.164314 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:30.166644 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:30.167311 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:30.167315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:30.167318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:30.167441 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:30.167754 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:30.167757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:30.167759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:30.168013 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:30.168134 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.168543 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.168769 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":51}
I20260807 03:02:30.173534 8148 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:33564:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 03:02:30.174489 8148 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:30.180557 8332 tablet_service.cc:1467] Processing CreateTablet for tablet 882f681c248e40ed9b814956cade1f52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96d8f9d1fe5c468bb26cb0edd05ec1b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:02:30.180517 8534 tablet_service.cc:1467] Processing CreateTablet for tablet 882f681c248e40ed9b814956cade1f52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96d8f9d1fe5c468bb26cb0edd05ec1b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:02:30.180716 8737 tablet_service.cc:1467] Processing CreateTablet for tablet 882f681c248e40ed9b814956cade1f52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96d8f9d1fe5c468bb26cb0edd05ec1b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:02:30.181490 8332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882f681c248e40ed9b814956cade1f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:30.181499 8737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882f681c248e40ed9b814956cade1f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:30.181574 8534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882f681c248e40ed9b814956cade1f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:30.187433 8847 tablet_bootstrap.cc:492] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: Bootstrap starting.
I20260807 03:02:30.188457 8848 tablet_bootstrap.cc:492] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: Bootstrap starting.
I20260807 03:02:30.188730 8851 tablet_bootstrap.cc:492] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Bootstrap starting.
I20260807 03:02:30.188894 8847 tablet_bootstrap.cc:654] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.190003 8851 tablet_bootstrap.cc:654] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.190059 8848 tablet_bootstrap.cc:654] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.192531 8847 tablet_bootstrap.cc:492] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: No bootstrap required, opened a new log
I20260807 03:02:30.192589 8848 tablet_bootstrap.cc:492] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: No bootstrap required, opened a new log
I20260807 03:02:30.192662 8847 ts_tablet_manager.cc:1397] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260807 03:02:30.192737 8848 ts_tablet_manager.cc:1397] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260807 03:02:30.193267 8847 raft_consensus.cc:348] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.193468 8847 raft_consensus.cc:374] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.193576 8847 raft_consensus.cc:729] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064a4f51415d47ab9bc5fdf39943172a, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.193532 8848 raft_consensus.cc:348] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.193782 8848 raft_consensus.cc:374] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.193885 8848 raft_consensus.cc:729] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.193800 8847 consensus_queue.cc:260] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.194118 8848 consensus_queue.cc:260] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.194208 8856 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.194507 8856 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:30.194741 8848 ts_tablet_manager.cc:1428] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 03:02:30.194938 8847 ts_tablet_manager.cc:1428] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 03:02:30.194811 8853 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.195092 8853 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:30.202034 8851 tablet_bootstrap.cc:492] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: No bootstrap required, opened a new log
I20260807 03:02:30.202158 8851 ts_tablet_manager.cc:1397] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.012s
I20260807 03:02:30.202827 8851 raft_consensus.cc:348] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.203019 8851 raft_consensus.cc:374] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.203085 8851 raft_consensus.cc:729] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163f3e6f66504b3e8f5b02dfdf3bdf48, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.203255 8851 consensus_queue.cc:260] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.203920 8851 ts_tablet_manager.cc:1428] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 03:02:30.203747 8859 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.204130 8859 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:30.468294 8859 raft_consensus.cc:482] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:30.468510 8859 raft_consensus.cc:504] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.470069 8859 leader_election.cc:288] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711)
I20260807 03:02:30.478682 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.470766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:37119 (local address 127.5.122.1:33515)
0807 03:02:30.470997 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.471005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.471030 (+ 25us) server_negotiation.cc:407] Connection header received
0807 03:02:30.471120 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.471125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.471228 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.471338 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:30.472024 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.473242 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.476030 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.477348 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.477449 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.477564 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.478216 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.478385 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 03:02:30.479640 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.470305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:33515 (local address 127.5.122.3:37119)
0807 03:02:30.470770 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.470791 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.470945 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.471539 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.471544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.471578 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:30.471808 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.471820 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.473439 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.473444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.475821 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.475839 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.477745 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.477752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.477980 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.478042 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.479148 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.479304 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":493,"threads_started":1}
I20260807 03:02:30.480402 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a" is_pre_election: true
I20260807 03:02:30.480679 8363 raft_consensus.cc:2454] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163f3e6f66504b3e8f5b02dfdf3bdf48 in term 0.
I20260807 03:02:30.481127 8663 leader_election.cc:302] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064a4f51415d47ab9bc5fdf39943172a, 163f3e6f66504b3e8f5b02dfdf3bdf48; no voters:
I20260807 03:02:30.481518 8859 raft_consensus.cc:2781] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:30.481662 8859 raft_consensus.cc:482] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:30.481753 8859 raft_consensus.cc:3037] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:30.483186 8853 raft_consensus.cc:482] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:30.483426 8853 raft_consensus.cc:504] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.484488 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.471362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:52809 (local address 127.5.122.2:34711)
0807 03:02:30.471553 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.471559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.478363 (+ 6804us) server_negotiation.cc:407] Connection header received
0807 03:02:30.478474 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.478480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.478561 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.478638 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:30.479257 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.480202 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.482205 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.483363 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.483515 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.483625 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.483973 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.484167 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 03:02:30.484480 8859 raft_consensus.cc:504] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.484815 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" is_pre_election: true
I20260807 03:02:30.484891 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a" is_pre_election: true
I20260807 03:02:30.485169 8766 raft_consensus.cc:2379] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 831e70d9006f494ba1a6987f7287f9bc in current term 1: Already voted for candidate 163f3e6f66504b3e8f5b02dfdf3bdf48 in this term.
I20260807 03:02:30.485139 8363 raft_consensus.cc:2454] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 0.
I20260807 03:02:30.485317 8859 leader_election.cc:288] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 election: Requested vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711)
I20260807 03:02:30.485852 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.471218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:34711 (local address 127.5.122.3:52809)
0807 03:02:30.478206 (+ 6988us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.478221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.478358 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.478817 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.478822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.478850 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:30.479106 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.479117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.480324 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.480330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.482059 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.482073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.483591 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.483597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.483795 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.483828 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.483916 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.484093 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6891,"thread_start_us":6771,"threads_started":1}
I20260807 03:02:30.485749 8465 leader_election.cc:302] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 064a4f51415d47ab9bc5fdf39943172a, 831e70d9006f494ba1a6987f7287f9bc; no voters: 163f3e6f66504b3e8f5b02dfdf3bdf48
I20260807 03:02:30.485800 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a"
I20260807 03:02:30.485997 8363 raft_consensus.cc:3037] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:30.486143 8868 raft_consensus.cc:2781] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:30.486279 8868 raft_consensus.cc:482] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:30.486416 8868 raft_consensus.cc:3037] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:30.488180 8363 raft_consensus.cc:2454] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163f3e6f66504b3e8f5b02dfdf3bdf48 in term 1.
I20260807 03:02:30.488565 8663 leader_election.cc:302] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064a4f51415d47ab9bc5fdf39943172a, 163f3e6f66504b3e8f5b02dfdf3bdf48; no voters:
I20260807 03:02:30.488845 8859 raft_consensus.cc:2781] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:30.488729 8868 raft_consensus.cc:504] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.489341 8859 raft_consensus.cc:686] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 LEADER]: Becoming Leader. State: Replica: 163f3e6f66504b3e8f5b02dfdf3bdf48, State: Running, Role: LEADER
I20260807 03:02:30.489840 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a"
I20260807 03:02:30.484241 8853 leader_election.cc:288] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:30.489886 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
I20260807 03:02:30.490157 8868 leader_election.cc:288] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Requested vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:30.490018 8859 consensus_queue.cc:237] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.490864 8567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831e70d9006f494ba1a6987f7287f9bc" is_pre_election: true
I20260807 03:02:30.491153 8567 raft_consensus.cc:2379] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 163f3e6f66504b3e8f5b02dfdf3bdf48 in current term 1: Already voted for candidate 831e70d9006f494ba1a6987f7287f9bc in this term.
I20260807 03:02:30.491063 8566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882f681c248e40ed9b814956cade1f52" candidate_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831e70d9006f494ba1a6987f7287f9bc"
I20260807 03:02:30.491377 8566 raft_consensus.cc:2379] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 163f3e6f66504b3e8f5b02dfdf3bdf48 in current term 1: Already voted for candidate 831e70d9006f494ba1a6987f7287f9bc in this term.
I20260807 03:02:30.491827 8363 raft_consensus.cc:2379] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 831e70d9006f494ba1a6987f7287f9bc in current term 1: Already voted for candidate 163f3e6f66504b3e8f5b02dfdf3bdf48 in this term.
I20260807 03:02:30.492211 8465 leader_election.cc:302] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 831e70d9006f494ba1a6987f7287f9bc; no voters: 064a4f51415d47ab9bc5fdf39943172a, 163f3e6f66504b3e8f5b02dfdf3bdf48
I20260807 03:02:30.492583 8868 raft_consensus.cc:2726] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 03:02:30.493966 8887 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: TxnStatusTablet state changed. Reason: New leader 163f3e6f66504b3e8f5b02dfdf3bdf48. Latest consensus state: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.494362 8887 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:02:30.495376 8889 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 03:02:30.495601 8889 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 03:02:30.495831 8145 catalog_manager.cc:5495] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } health_report { overall_health: HEALTHY } } }
I20260807 03:02:30.560379 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.555060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:44445 (local address 127.0.0.1:59124)
0807 03:02:30.555282 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.555304 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.555448 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.555895 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.555900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.555913 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:30.556094 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.556103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.557089 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.557095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.558086 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.558099 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.558848 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.558854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.558997 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.559049 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:30.559851 (+ 802us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:30.559874 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.559953 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.560098 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 03:02:30.560379 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.555203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.5.122.3:44445)
0807 03:02:30.555389 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.555395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.555418 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:02:30.555652 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.555656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.555735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.555866 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:30.556224 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.556957 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.558223 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.558696 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.558806 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.559227 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:30.559742 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:30.559891 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.559973 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.560057 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 03:02:30.567880 8566 raft_consensus.cc:1261] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Refusing update from remote peer 163f3e6f66504b3e8f5b02dfdf3bdf48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:30.568074 8363 raft_consensus.cc:1261] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Refusing update from remote peer 163f3e6f66504b3e8f5b02dfdf3bdf48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:30.568706 8887 consensus_queue.cc:1035] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:30.569073 8859 consensus_queue.cc:1035] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:30.577481 8868 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: TxnStatusTablet state changed. Reason: New leader 163f3e6f66504b3e8f5b02dfdf3bdf48. Latest consensus state: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.577772 8868 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:30.580789 8893 mvcc.cc:205] Tried to move back new op lower bound from 7315749890317160448 to 7315749890012475392. Current Snapshot: MvccSnapshot[applied={T|T < 7315749890317160448}]
I20260807 03:02:30.581590 8856 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: TxnStatusTablet state changed. Reason: New leader 163f3e6f66504b3e8f5b02dfdf3bdf48. Latest consensus state: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.581648 8887 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.581894 8856 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:30.581964 8887 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:02:30.585796 8868 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.586091 8868 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 831e70d9006f494ba1a6987f7287f9bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:30.589298 8887 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.589645 8887 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 163f3e6f66504b3e8f5b02dfdf3bdf48: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:02:30.591276 8898 mvcc.cc:205] Tried to move back new op lower bound from 7315749890317160448 to 7315749890012475392. Current Snapshot: MvccSnapshot[applied={T|T < 7315749890317160448}]
I20260807 03:02:30.599953 8856 tablet_replica.cc:439] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } } }
I20260807 03:02:30.600220 8856 tablet_replica.cc:442] T 882f681c248e40ed9b814956cade1f52 P 064a4f51415d47ab9bc5fdf39943172a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:02:30.606424 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.599881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.5.122.2:34711)
0807 03:02:30.600041 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.600046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.600066 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:02:30.600488 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.600495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.600608 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.600751 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:30.601456 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.602552 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.603962 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.604608 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.604733 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.605144 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:30.605800 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:30.605933 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.606036 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.606122 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:02:30.621404 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.616394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:45993 (local address 127.5.122.3:44445)
0807 03:02:30.616583 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.616588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.616913 (+ 325us) server_negotiation.cc:407] Connection header received
0807 03:02:30.617082 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.617086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.617153 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.617231 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:30.617635 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.618473 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.619759 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.620192 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.620303 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.620713 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:30.620819 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:30.620894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.621034 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.621121 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:02:30.621470 8912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.616282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:44445 (local address 127.5.122.2:45993)
0807 03:02:30.616802 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.616821 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.616962 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.617253 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.617257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.617269 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:30.617502 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.617510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.618591 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.618597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.619624 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.619636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.620334 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.620339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.620469 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.620528 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:30.620922 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:30.620933 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.620996 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.621145 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":256,"threads_started":1}
I20260807 03:02:30.630623 8363 raft_consensus.cc:1261] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Refusing update from remote peer 831e70d9006f494ba1a6987f7287f9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:30.630754 8766 raft_consensus.cc:1261] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Refusing update from remote peer 831e70d9006f494ba1a6987f7287f9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:30.631244 8853 consensus_queue.cc:1035] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:30.631606 8868 consensus_queue.cc:1035] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:30.638051 8896 mvcc.cc:205] Tried to move back new op lower bound from 7315749890576871424 to 7315749888426655744. Current Snapshot: MvccSnapshot[applied={T|T < 7315749890576871424}]
I20260807 03:02:30.672214 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.665227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:44401 (local address 127.5.122.2:34711)
0807 03:02:30.665400 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:30.665406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:30.666532 (+ 1126us) server_negotiation.cc:407] Connection header received
0807 03:02:30.666763 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:30.666769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:30.666863 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:30.666922 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:30.667625 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.668387 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.670592 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.671128 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.671225 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.671654 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:30.671751 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:30.671812 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:30.671951 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 03:02:30.672007 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 03:02:30.672516 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:30.665110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:34711 (local address 127.5.122.3:44401)
0807 03:02:30.666426 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:30.666450 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:30.666588 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:30.667102 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:30.667107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:30.667119 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:30.667465 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.667482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.668527 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.668533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.670418 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:30.670434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:30.671261 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:30.671267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:30.671434 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:30.671504 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:30.671840 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:30.671858 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 03:02:30.671940 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 03:02:30.672084 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":931,"threads_started":1}
I20260807 03:02:30.708360 8145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:30.708665 8145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:30.712608 8145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Sending DeleteTablet for 3 replicas of tablet 0f0a2cfed3d94ebf8c8a34c5e774675c
I20260807 03:02:30.713634 8332 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:30 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59440
I20260807 03:02:30.714610 8534 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:30 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:41628
I20260807 03:02:30.715062 8737 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f0a2cfed3d94ebf8c8a34c5e774675c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:30 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59110
I20260807 03:02:30.715442 8924 tablet_replica.cc:330] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: stopping tablet replica
I20260807 03:02:30.715873 8924 raft_consensus.cc:2229] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:30.716521 8924 raft_consensus.cc:2258] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:30.716727 8145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:30.717406 8145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:30.718616 8924 ts_tablet_manager.cc:1905] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:30.722606 8925 tablet_replica.cc:330] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: stopping tablet replica
I20260807 03:02:30.723140 8925 raft_consensus.cc:2229] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:30.723476 8925 raft_consensus.cc:2258] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:30.725651 8331 tablet_service.cc:1467] Processing CreateTablet for tablet 41014919d6e84806a2bc79638cbd1c32 (DEFAULT_TABLE table=test-table [id=447b47ffc3d84174af0f5bddaebc54d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:30.725950 8534 tablet_service.cc:1467] Processing CreateTablet for tablet 41014919d6e84806a2bc79638cbd1c32 (DEFAULT_TABLE table=test-table [id=447b47ffc3d84174af0f5bddaebc54d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:30.726089 8331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41014919d6e84806a2bc79638cbd1c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:30.726367 8534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41014919d6e84806a2bc79638cbd1c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:30.734102 8925 ts_tablet_manager.cc:1905] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:30.736837 8924 ts_tablet_manager.cc:1918] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:30.736570 8737 tablet_service.cc:1467] Processing CreateTablet for tablet 41014919d6e84806a2bc79638cbd1c32 (DEFAULT_TABLE table=test-table [id=447b47ffc3d84174af0f5bddaebc54d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:30.737412 8737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41014919d6e84806a2bc79638cbd1c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:30.749327 8924 log.cc:1196] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/0f0a2cfed3d94ebf8c8a34c5e774675c
I20260807 03:02:30.750149 8924 ts_tablet_manager.cc:1939] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 831e70d9006f494ba1a6987f7287f9bc: Deleting consensus metadata
I20260807 03:02:30.754768 8929 tablet_bootstrap.cc:492] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Bootstrap starting.
I20260807 03:02:30.755504 8926 tablet_bootstrap.cc:492] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: Bootstrap starting.
I20260807 03:02:30.756186 8928 tablet_bootstrap.cc:492] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: Bootstrap starting.
I20260807 03:02:30.756561 8929 tablet_bootstrap.cc:654] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.756999 8130 catalog_manager.cc:4841] TS 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711): tablet 0f0a2cfed3d94ebf8c8a34c5e774675c (table test-table [id=8108e7e65e3e4364b9806b9acb38d5de]) successfully deleted
I20260807 03:02:30.759167 8926 tablet_bootstrap.cc:654] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.761935 8928 tablet_bootstrap.cc:654] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:30.763535 8925 ts_tablet_manager.cc:1918] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:30.763742 8925 log.cc:1196] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/0f0a2cfed3d94ebf8c8a34c5e774675c
I20260807 03:02:30.764412 8925 ts_tablet_manager.cc:1939] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting consensus metadata
I20260807 03:02:30.766165 8130 catalog_manager.cc:4841] TS 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445): tablet 0f0a2cfed3d94ebf8c8a34c5e774675c (table test-table [id=8108e7e65e3e4364b9806b9acb38d5de]) successfully deleted
I20260807 03:02:30.767867 8929 tablet_bootstrap.cc:492] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: No bootstrap required, opened a new log
I20260807 03:02:30.768009 8929 ts_tablet_manager.cc:1397] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260807 03:02:30.768064 8923 tablet_replica.cc:330] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: stopping tablet replica
I20260807 03:02:30.768496 8923 raft_consensus.cc:2229] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:30.768895 8923 raft_consensus.cc:2258] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:30.768795 8929 raft_consensus.cc:348] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.769058 8929 raft_consensus.cc:374] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.769150 8929 raft_consensus.cc:729] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163f3e6f66504b3e8f5b02dfdf3bdf48, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.769436 8929 consensus_queue.cc:260] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.770303 8928 tablet_bootstrap.cc:492] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: No bootstrap required, opened a new log
I20260807 03:02:30.770491 8928 ts_tablet_manager.cc:1397] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260807 03:02:30.771265 8928 raft_consensus.cc:348] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.771502 8928 raft_consensus.cc:374] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.771608 8928 raft_consensus.cc:729] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.772513 8929 ts_tablet_manager.cc:1428] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 03:02:30.773437 8923 ts_tablet_manager.cc:1905] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:30.774104 8928 consensus_queue.cc:260] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.774832 8928 ts_tablet_manager.cc:1428] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260807 03:02:30.779568 8926 tablet_bootstrap.cc:492] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: No bootstrap required, opened a new log
I20260807 03:02:30.779776 8926 ts_tablet_manager.cc:1397] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.007s
I20260807 03:02:30.780601 8926 raft_consensus.cc:348] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.780922 8926 raft_consensus.cc:374] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:30.781067 8926 raft_consensus.cc:729] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064a4f51415d47ab9bc5fdf39943172a, State: Initialized, Role: FOLLOWER
I20260807 03:02:30.781374 8926 consensus_queue.cc:260] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:30.782655 8926 ts_tablet_manager.cc:1428] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 03:02:30.783179 8923 ts_tablet_manager.cc:1918] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:30.783373 8923 log.cc:1196] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/0f0a2cfed3d94ebf8c8a34c5e774675c
I20260807 03:02:30.783983 8923 ts_tablet_manager.cc:1939] T 0f0a2cfed3d94ebf8c8a34c5e774675c P 064a4f51415d47ab9bc5fdf39943172a: Deleting consensus metadata
I20260807 03:02:30.786118 8130 catalog_manager.cc:4841] TS 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515): tablet 0f0a2cfed3d94ebf8c8a34c5e774675c (table test-table [id=8108e7e65e3e4364b9806b9acb38d5de]) successfully deleted
I20260807 03:02:31.120977 8921 raft_consensus.cc:482] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:31.121439 8921 raft_consensus.cc:504] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:31.122691 8921 leader_election.cc:288] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:31.123329 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41014919d6e84806a2bc79638cbd1c32" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" is_pre_election: true
I20260807 03:02:31.123354 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41014919d6e84806a2bc79638cbd1c32" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a" is_pre_election: true
I20260807 03:02:31.123588 8766 raft_consensus.cc:2454] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 0.
I20260807 03:02:31.123704 8363 raft_consensus.cc:2454] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 0.
I20260807 03:02:31.124137 8465 leader_election.cc:302] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163f3e6f66504b3e8f5b02dfdf3bdf48, 831e70d9006f494ba1a6987f7287f9bc; no voters:
I20260807 03:02:31.124451 8921 raft_consensus.cc:2781] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:31.124599 8921 raft_consensus.cc:482] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:31.124732 8921 raft_consensus.cc:3037] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:31.127507 8921 raft_consensus.cc:504] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:31.128254 8921 leader_election.cc:288] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Requested vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:31.128568 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41014919d6e84806a2bc79638cbd1c32" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a"
I20260807 03:02:31.128845 8363 raft_consensus.cc:3037] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:31.128821 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41014919d6e84806a2bc79638cbd1c32" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
I20260807 03:02:31.129069 8766 raft_consensus.cc:3037] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:31.131667 8766 raft_consensus.cc:2454] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 1.
I20260807 03:02:31.132052 8363 raft_consensus.cc:2454] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 1.
I20260807 03:02:31.132117 8465 leader_election.cc:302] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163f3e6f66504b3e8f5b02dfdf3bdf48, 831e70d9006f494ba1a6987f7287f9bc; no voters:
I20260807 03:02:31.132436 8921 raft_consensus.cc:2781] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:31.132634 8921 raft_consensus.cc:686] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 LEADER]: Becoming Leader. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Running, Role: LEADER
I20260807 03:02:31.132977 8921 consensus_queue.cc:237] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:31.136433 8145 catalog_manager.cc:5495] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2). New cstate: current_term: 1 leader_uuid: "831e70d9006f494ba1a6987f7287f9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } health_report { overall_health: UNKNOWN } } }
I20260807 03:02:31.217072 8766 raft_consensus.cc:1261] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Refusing update from remote peer 831e70d9006f494ba1a6987f7287f9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:31.217162 8363 raft_consensus.cc:1261] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Refusing update from remote peer 831e70d9006f494ba1a6987f7287f9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:31.217588 8921 consensus_queue.cc:1035] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:31.217895 8868 consensus_queue.cc:1035] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 03:02:31.239033 8938 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 03:02:32.832314 8145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:32.832584 8145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:32.837055 8145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Sending DeleteTablet for 3 replicas of tablet 41014919d6e84806a2bc79638cbd1c32
I20260807 03:02:32.837775 8332 tablet_service.cc:1514] Processing DeleteTablet for tablet 41014919d6e84806a2bc79638cbd1c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:32 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59440
I20260807 03:02:32.837834 8534 tablet_service.cc:1514] Processing DeleteTablet for tablet 41014919d6e84806a2bc79638cbd1c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:32 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:41628
I20260807 03:02:32.838126 8737 tablet_service.cc:1514] Processing DeleteTablet for tablet 41014919d6e84806a2bc79638cbd1c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:32 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59110
I20260807 03:02:32.849522 8986 tablet_replica.cc:330] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: stopping tablet replica
I20260807 03:02:32.849648 8989 tablet_replica.cc:330] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: stopping tablet replica
I20260807 03:02:32.849871 8989 raft_consensus.cc:2229] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:32.849951 8986 raft_consensus.cc:2229] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:32.850183 8989 raft_consensus.cc:2258] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:32.850543 8986 raft_consensus.cc:2258] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:32.851320 8989 ts_tablet_manager.cc:1905] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:32.851729 8986 ts_tablet_manager.cc:1905] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:32.852716 8988 tablet_replica.cc:330] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: stopping tablet replica
I20260807 03:02:32.853042 8988 raft_consensus.cc:2229] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:32.853560 8988 raft_consensus.cc:2258] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:32.854576 8988 ts_tablet_manager.cc:1905] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:32.858944 8989 ts_tablet_manager.cc:1918] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 03:02:32.859143 8989 log.cc:1196] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/41014919d6e84806a2bc79638cbd1c32
I20260807 03:02:32.859177 8986 ts_tablet_manager.cc:1918] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 03:02:32.859313 8986 log.cc:1196] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/41014919d6e84806a2bc79638cbd1c32
I20260807 03:02:32.859669 8989 ts_tablet_manager.cc:1939] T 41014919d6e84806a2bc79638cbd1c32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting consensus metadata
I20260807 03:02:32.859751 8986 ts_tablet_manager.cc:1939] T 41014919d6e84806a2bc79638cbd1c32 P 064a4f51415d47ab9bc5fdf39943172a: Deleting consensus metadata
I20260807 03:02:32.860877 8130 catalog_manager.cc:4841] TS 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515): tablet 41014919d6e84806a2bc79638cbd1c32 (table test-table [id=447b47ffc3d84174af0f5bddaebc54d5]) successfully deleted
I20260807 03:02:32.861132 8988 ts_tablet_manager.cc:1918] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 03:02:32.861141 8130 catalog_manager.cc:4841] TS 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445): tablet 41014919d6e84806a2bc79638cbd1c32 (table test-table [id=447b47ffc3d84174af0f5bddaebc54d5]) successfully deleted
I20260807 03:02:32.861255 8988 log.cc:1196] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/41014919d6e84806a2bc79638cbd1c32
I20260807 03:02:32.861666 8988 ts_tablet_manager.cc:1939] T 41014919d6e84806a2bc79638cbd1c32 P 831e70d9006f494ba1a6987f7287f9bc: Deleting consensus metadata
I20260807 03:02:32.862865 8130 catalog_manager.cc:4841] TS 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711): tablet 41014919d6e84806a2bc79638cbd1c32 (table test-table [id=447b47ffc3d84174af0f5bddaebc54d5]) successfully deleted
Aug 07 03:02:32 dist-test-slave-fgb3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 03:02:32 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:32.901610 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:32.884458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.5.122.62:33883)
0807 03:02:32.884974 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:32.884979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:32.885076 (+ 97us) server_negotiation.cc:407] Connection header received
0807 03:02:32.885231 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:32.885235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:32.885300 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:32.885389 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:32.886932 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:32.888005 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:32.889774 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:32.890444 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:32.890566 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:32.895376 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:32.895383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:32.895388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:32.895428 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:32.898717 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:32.899448 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:32.899453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:32.899457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:32.899601 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:32.899981 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:32.899986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:32.899989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:32.900302 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:32.900882 (+ 580us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:32.901010 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 03:02:32.901223 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":234,"threads_started":1}
I20260807 03:02:32.905680 8145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:32.906383 8145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:32.914184 8332 tablet_service.cc:1467] Processing CreateTablet for tablet 37e795891e134fd08aff3482dcd16f32 (DEFAULT_TABLE table=test-table [id=4301c10421514a4d8e0a45acd6be93bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:32.914189 8737 tablet_service.cc:1467] Processing CreateTablet for tablet 37e795891e134fd08aff3482dcd16f32 (DEFAULT_TABLE table=test-table [id=4301c10421514a4d8e0a45acd6be93bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:32.914690 8737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e795891e134fd08aff3482dcd16f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:32.914711 8332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e795891e134fd08aff3482dcd16f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:32.918856 8534 tablet_service.cc:1467] Processing CreateTablet for tablet 37e795891e134fd08aff3482dcd16f32 (DEFAULT_TABLE table=test-table [id=4301c10421514a4d8e0a45acd6be93bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:32.919342 8534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e795891e134fd08aff3482dcd16f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:32.920789 9003 tablet_bootstrap.cc:492] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Bootstrap starting.
I20260807 03:02:32.922851 9003 tablet_bootstrap.cc:654] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:32.928045 9004 tablet_bootstrap.cc:492] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: Bootstrap starting.
I20260807 03:02:32.928293 9006 tablet_bootstrap.cc:492] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: Bootstrap starting.
I20260807 03:02:32.929299 9003 tablet_bootstrap.cc:492] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: No bootstrap required, opened a new log
I20260807 03:02:32.929477 9003 ts_tablet_manager.cc:1397] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 03:02:32.930353 9004 tablet_bootstrap.cc:654] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:32.930605 9006 tablet_bootstrap.cc:654] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:32.930516 9003 raft_consensus.cc:348] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.930837 9003 raft_consensus.cc:374] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:32.931015 9003 raft_consensus.cc:729] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163f3e6f66504b3e8f5b02dfdf3bdf48, State: Initialized, Role: FOLLOWER
I20260807 03:02:32.931324 9003 consensus_queue.cc:260] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.931972 9003 ts_tablet_manager.cc:1428] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 03:02:32.935618 8958 raft_consensus.cc:482] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:32.935827 8958 raft_consensus.cc:504] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.936560 8958 leader_election.cc:288] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711)
I20260807 03:02:32.937196 8362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e795891e134fd08aff3482dcd16f32" candidate_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a" is_pre_election: true
I20260807 03:02:32.937278 8566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e795891e134fd08aff3482dcd16f32" candidate_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831e70d9006f494ba1a6987f7287f9bc" is_pre_election: true
W20260807 03:02:32.938170 8663 leader_election.cc:341] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515): Illegal state: must be running to vote when last-logged opid is not known
W20260807 03:02:32.938542 8663 leader_election.cc:341] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711): Illegal state: must be running to vote when last-logged opid is not known
I20260807 03:02:32.938694 8663 leader_election.cc:302] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 163f3e6f66504b3e8f5b02dfdf3bdf48; no voters: 064a4f51415d47ab9bc5fdf39943172a, 831e70d9006f494ba1a6987f7287f9bc
I20260807 03:02:32.939064 8958 raft_consensus.cc:2726] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 03:02:32.939612 9004 tablet_bootstrap.cc:492] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: No bootstrap required, opened a new log
I20260807 03:02:32.939697 9006 tablet_bootstrap.cc:492] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: No bootstrap required, opened a new log
I20260807 03:02:32.939725 9004 ts_tablet_manager.cc:1397] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 03:02:32.939837 9006 ts_tablet_manager.cc:1397] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260807 03:02:32.940440 9004 raft_consensus.cc:348] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.940704 9004 raft_consensus.cc:374] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:32.940637 9006 raft_consensus.cc:348] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.940800 9004 raft_consensus.cc:729] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064a4f51415d47ab9bc5fdf39943172a, State: Initialized, Role: FOLLOWER
I20260807 03:02:32.940868 9006 raft_consensus.cc:374] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:32.940975 9006 raft_consensus.cc:729] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Initialized, Role: FOLLOWER
I20260807 03:02:32.941118 9004 consensus_queue.cc:260] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.941212 9006 consensus_queue.cc:260] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.941795 9006 ts_tablet_manager.cc:1428] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 03:02:32.942144 9004 ts_tablet_manager.cc:1428] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 03:02:32.964941 9010 raft_consensus.cc:482] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:32.965181 9010 raft_consensus.cc:504] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.966020 9010 leader_election.cc:288] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:32.966614 8566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e795891e134fd08aff3482dcd16f32" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831e70d9006f494ba1a6987f7287f9bc" is_pre_election: true
I20260807 03:02:32.966609 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e795891e134fd08aff3482dcd16f32" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" is_pre_election: true
I20260807 03:02:32.966857 8566 raft_consensus.cc:2454] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 064a4f51415d47ab9bc5fdf39943172a in term 0.
I20260807 03:02:32.966881 8766 raft_consensus.cc:2454] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 064a4f51415d47ab9bc5fdf39943172a in term 0.
I20260807 03:02:32.967236 8264 leader_election.cc:302] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064a4f51415d47ab9bc5fdf39943172a, 831e70d9006f494ba1a6987f7287f9bc; no voters:
I20260807 03:02:32.967486 9010 raft_consensus.cc:2781] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:32.967599 9010 raft_consensus.cc:482] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:32.967720 9010 raft_consensus.cc:3037] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:32.970037 9010 raft_consensus.cc:504] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.970666 9010 leader_election.cc:288] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 election: Requested vote from peers 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:32.971129 8566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e795891e134fd08aff3482dcd16f32" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831e70d9006f494ba1a6987f7287f9bc"
I20260807 03:02:32.971180 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e795891e134fd08aff3482dcd16f32" candidate_uuid: "064a4f51415d47ab9bc5fdf39943172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
I20260807 03:02:32.971370 8766 raft_consensus.cc:3037] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:32.971380 8566 raft_consensus.cc:3037] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:32.973517 8766 raft_consensus.cc:2454] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 064a4f51415d47ab9bc5fdf39943172a in term 1.
I20260807 03:02:32.973682 8566 raft_consensus.cc:2454] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 064a4f51415d47ab9bc5fdf39943172a in term 1.
I20260807 03:02:32.973870 8264 leader_election.cc:302] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 064a4f51415d47ab9bc5fdf39943172a, 163f3e6f66504b3e8f5b02dfdf3bdf48; no voters:
I20260807 03:02:32.974136 9010 raft_consensus.cc:2781] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:32.974750 9010 raft_consensus.cc:686] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 LEADER]: Becoming Leader. State: Replica: 064a4f51415d47ab9bc5fdf39943172a, State: Running, Role: LEADER
I20260807 03:02:32.975214 9010 consensus_queue.cc:237] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:32.978853 8148 catalog_manager.cc:5495] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a reported cstate change: term changed from 0 to 1, leader changed from <none> to 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1). New cstate: current_term: 1 leader_uuid: "064a4f51415d47ab9bc5fdf39943172a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } health_report { overall_health: UNKNOWN } } }
I20260807 03:02:33.002161 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:32.996405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.5.122.1:33515)
0807 03:02:32.996987 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:32.996992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:32.997015 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:02:32.997099 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:32.997102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:32.997189 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:32.997341 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:32.997666 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:32.998615 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.000019 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.000438 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.000528 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:33.000969 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:33.001570 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:33.001683 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:33.001740 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 03:02:33.001811 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":321,"threads_started":1}
I20260807 03:02:33.017899 9019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:33.011833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:44445 (local address 127.5.122.1:56853)
0807 03:02:33.012382 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:33.012397 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:33.012497 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:33.013109 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:33.013111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:33.013120 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:33.013481 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:33.013490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.014695 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.014699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:33.015921 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:33.015933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.016704 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.016708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:33.016829 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:33.016879 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:33.017241 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:33.017266 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 03:02:33.017387 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 03:02:33.017534 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":274,"threads_started":1}
I20260807 03:02:33.017910 9020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:33.012099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:56853 (local address 127.5.122.3:44445)
0807 03:02:33.012869 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:33.012874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:33.012895 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:02:33.012955 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:33.012958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:33.013019 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:33.013095 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:33.013679 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.014535 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.016027 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.016545 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.016635 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:33.017032 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:33.017131 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:33.017198 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:33.017447 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 03:02:33.017511 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":455,"threads_started":1}
I20260807 03:02:33.026412 8566 raft_consensus.cc:1261] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Refusing update from remote peer 064a4f51415d47ab9bc5fdf39943172a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:33.026621 8766 raft_consensus.cc:1261] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Refusing update from remote peer 064a4f51415d47ab9bc5fdf39943172a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:33.027069 9012 consensus_queue.cc:1035] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [LEADER]: Connected to new peer: Peer: permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:33.027427 9010 consensus_queue.cc:1035] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [LEADER]: Connected to new peer: Peer: permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:33.055132 9031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:33.049168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:33515 (local address 127.5.122.3:42153)
0807 03:02:33.049800 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:33.049815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:33.049919 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:33.050131 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:33.050134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:33.050141 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:02:33.050451 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:33.050462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.051432 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.051438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:33.053121 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:33.053138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.053975 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.053983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:33.054141 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:33.054180 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:02:33.054540 (+ 360us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:02:33.054552 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 03:02:33.054622 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 03:02:33.054749 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":371,"threads_started":1}
I20260807 03:02:33.055133 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:33.049202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:42153 (local address 127.5.122.1:33515)
0807 03:02:33.049382 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:33.049388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:33.049876 (+ 488us) server_negotiation.cc:407] Connection header received
0807 03:02:33.049998 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:33.050002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:33.050060 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:33.050124 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:33.050565 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.051308 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.053267 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.053767 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.053886 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:33.054350 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:33.054451 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:33.054524 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:33.054651 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 03:02:33.054725 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 03:02:33.085160 8144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:33.085489 8144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:33.089098 8144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Sending DeleteTablet for 3 replicas of tablet 37e795891e134fd08aff3482dcd16f32
I20260807 03:02:33.089982 8332 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e795891e134fd08aff3482dcd16f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:33 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59440
I20260807 03:02:33.090333 8986 tablet_replica.cc:330] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: stopping tablet replica
I20260807 03:02:33.090600 8986 raft_consensus.cc:2229] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:33.090947 8144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 03:02:33.091150 8986 raft_consensus.cc:2258] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 03:02:33.091691 8144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:33.092041 8737 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e795891e134fd08aff3482dcd16f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:33 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59110
I20260807 03:02:33.092233 8534 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e795891e134fd08aff3482dcd16f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:33 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:41628
I20260807 03:02:33.092401 8989 tablet_replica.cc:330] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: stopping tablet replica
I20260807 03:02:33.092617 8989 raft_consensus.cc:2229] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:33.092824 8989 raft_consensus.cc:2258] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:33.092849 8986 ts_tablet_manager.cc:1905] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:33.093175 8988 tablet_replica.cc:330] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: stopping tablet replica
I20260807 03:02:33.093408 8988 raft_consensus.cc:2229] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:33.093629 8988 raft_consensus.cc:2258] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:33.094832 8988 ts_tablet_manager.cc:1905] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:33.094981 8989 ts_tablet_manager.cc:1905] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:33.101382 8988 ts_tablet_manager.cc:1918] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:33.101565 8988 log.cc:1196] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/37e795891e134fd08aff3482dcd16f32
I20260807 03:02:33.102077 8988 ts_tablet_manager.cc:1939] T 37e795891e134fd08aff3482dcd16f32 P 831e70d9006f494ba1a6987f7287f9bc: Deleting consensus metadata
I20260807 03:02:33.102356 8989 ts_tablet_manager.cc:1918] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:33.102571 8989 log.cc:1196] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/37e795891e134fd08aff3482dcd16f32
I20260807 03:02:33.103055 8989 ts_tablet_manager.cc:1939] T 37e795891e134fd08aff3482dcd16f32 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting consensus metadata
I20260807 03:02:33.103231 8986 ts_tablet_manager.cc:1918] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:02:33.103447 8986 log.cc:1196] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/37e795891e134fd08aff3482dcd16f32
I20260807 03:02:33.103866 8986 ts_tablet_manager.cc:1939] T 37e795891e134fd08aff3482dcd16f32 P 064a4f51415d47ab9bc5fdf39943172a: Deleting consensus metadata
I20260807 03:02:33.104321 8130 catalog_manager.cc:4841] TS 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445): tablet 37e795891e134fd08aff3482dcd16f32 (table test-table [id=4301c10421514a4d8e0a45acd6be93bc]) successfully deleted
I20260807 03:02:33.104766 8332 tablet_service.cc:1467] Processing CreateTablet for tablet 024df21e840948f39fdbc18efc3d9947 (DEFAULT_TABLE table=test-table [id=b89978e50b37413ca67f2a6a70e92e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:33.104874 8130 catalog_manager.cc:4841] TS 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515): tablet 37e795891e134fd08aff3482dcd16f32 (table test-table [id=4301c10421514a4d8e0a45acd6be93bc]) successfully deleted
I20260807 03:02:33.105198 8332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024df21e840948f39fdbc18efc3d9947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:33.106595 8130 catalog_manager.cc:4841] TS 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711): tablet 37e795891e134fd08aff3482dcd16f32 (table test-table [id=4301c10421514a4d8e0a45acd6be93bc]) successfully deleted
I20260807 03:02:33.107095 8534 tablet_service.cc:1467] Processing CreateTablet for tablet 024df21e840948f39fdbc18efc3d9947 (DEFAULT_TABLE table=test-table [id=b89978e50b37413ca67f2a6a70e92e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:33.107110 8737 tablet_service.cc:1467] Processing CreateTablet for tablet 024df21e840948f39fdbc18efc3d9947 (DEFAULT_TABLE table=test-table [id=b89978e50b37413ca67f2a6a70e92e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:33.107527 8737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024df21e840948f39fdbc18efc3d9947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:33.107584 8534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024df21e840948f39fdbc18efc3d9947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:33.112239 9004 tablet_bootstrap.cc:492] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: Bootstrap starting.
I20260807 03:02:33.114279 9006 tablet_bootstrap.cc:492] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: Bootstrap starting.
I20260807 03:02:33.114611 9004 tablet_bootstrap.cc:654] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:33.116140 9006 tablet_bootstrap.cc:654] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:33.117322 9003 tablet_bootstrap.cc:492] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Bootstrap starting.
I20260807 03:02:33.119357 9003 tablet_bootstrap.cc:654] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:33.124717 9006 tablet_bootstrap.cc:492] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: No bootstrap required, opened a new log
I20260807 03:02:33.124934 9006 ts_tablet_manager.cc:1397] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 03:02:33.125722 9006 raft_consensus.cc:348] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.126008 9006 raft_consensus.cc:374] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:33.126109 9006 raft_consensus.cc:729] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Initialized, Role: FOLLOWER
I20260807 03:02:33.126461 9006 consensus_queue.cc:260] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.127240 9006 ts_tablet_manager.cc:1428] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 03:02:33.128846 9004 tablet_bootstrap.cc:492] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: No bootstrap required, opened a new log
I20260807 03:02:33.129050 9004 ts_tablet_manager.cc:1397] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.008s
I20260807 03:02:33.129992 9004 raft_consensus.cc:348] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.130326 9004 raft_consensus.cc:374] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:33.130430 9004 raft_consensus.cc:729] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064a4f51415d47ab9bc5fdf39943172a, State: Initialized, Role: FOLLOWER
I20260807 03:02:33.130822 9004 consensus_queue.cc:260] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.131417 9004 ts_tablet_manager.cc:1428] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 03:02:33.133061 9003 tablet_bootstrap.cc:492] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: No bootstrap required, opened a new log
I20260807 03:02:33.133246 9003 ts_tablet_manager.cc:1397] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20260807 03:02:33.134112 9003 raft_consensus.cc:348] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.134390 9003 raft_consensus.cc:374] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:33.134477 9003 raft_consensus.cc:729] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163f3e6f66504b3e8f5b02dfdf3bdf48, State: Initialized, Role: FOLLOWER
I20260807 03:02:33.134764 9003 consensus_queue.cc:260] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.135432 9003 ts_tablet_manager.cc:1428] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 03:02:33.323374 8976 raft_consensus.cc:482] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:33.323633 8976 raft_consensus.cc:504] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.324339 8976 leader_election.cc:288] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:33.325027 8362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024df21e840948f39fdbc18efc3d9947" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a" is_pre_election: true
I20260807 03:02:33.325102 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024df21e840948f39fdbc18efc3d9947" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" is_pre_election: true
I20260807 03:02:33.325335 8362 raft_consensus.cc:2454] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 0.
I20260807 03:02:33.325439 8766 raft_consensus.cc:2454] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 0.
I20260807 03:02:33.325943 8465 leader_election.cc:302] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163f3e6f66504b3e8f5b02dfdf3bdf48, 831e70d9006f494ba1a6987f7287f9bc; no voters:
I20260807 03:02:33.326399 8976 raft_consensus.cc:2781] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:33.326553 8976 raft_consensus.cc:482] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:33.326671 8976 raft_consensus.cc:3037] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:33.329564 8976 raft_consensus.cc:504] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.330186 8976 leader_election.cc:288] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Requested vote from peers 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515), 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445)
I20260807 03:02:33.330794 8362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024df21e840948f39fdbc18efc3d9947" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064a4f51415d47ab9bc5fdf39943172a"
I20260807 03:02:33.330768 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024df21e840948f39fdbc18efc3d9947" candidate_uuid: "831e70d9006f494ba1a6987f7287f9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48"
I20260807 03:02:33.331046 8766 raft_consensus.cc:3037] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:33.331069 8362 raft_consensus.cc:3037] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:33.334376 8766 raft_consensus.cc:2454] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 1.
I20260807 03:02:33.334395 8362 raft_consensus.cc:2454] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831e70d9006f494ba1a6987f7287f9bc in term 1.
I20260807 03:02:33.334764 8465 leader_election.cc:302] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163f3e6f66504b3e8f5b02dfdf3bdf48, 831e70d9006f494ba1a6987f7287f9bc; no voters:
I20260807 03:02:33.335117 8976 raft_consensus.cc:2781] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:33.335768 8976 raft_consensus.cc:686] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 1 LEADER]: Becoming Leader. State: Replica: 831e70d9006f494ba1a6987f7287f9bc, State: Running, Role: LEADER
I20260807 03:02:33.336169 8976 consensus_queue.cc:237] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } }
I20260807 03:02:33.338904 8148 catalog_manager.cc:5495] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2). New cstate: current_term: 1 leader_uuid: "831e70d9006f494ba1a6987f7287f9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "831e70d9006f494ba1a6987f7287f9bc" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 } health_report { overall_health: UNKNOWN } } }
I20260807 03:02:33.374083 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:33.367795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.5.122.2:34711)
0807 03:02:33.368511 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:33.368515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:33.368550 (+ 35us) server_negotiation.cc:407] Connection header received
0807 03:02:33.368601 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:33.368604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:33.368663 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:33.368739 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:33.369243 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.370192 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.372185 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:33.372607 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:33.372695 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:33.373301 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:33.373413 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:33.373480 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:33.373637 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 03:02:33.373696 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":434,"threads_started":1}
I20260807 03:02:33.383919 8766 raft_consensus.cc:1261] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Refusing update from remote peer 831e70d9006f494ba1a6987f7287f9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:33.384486 8362 raft_consensus.cc:1261] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Refusing update from remote peer 831e70d9006f494ba1a6987f7287f9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:33.384501 8976 consensus_queue.cc:1035] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "163f3e6f66504b3e8f5b02dfdf3bdf48" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 44445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:33.385006 9038 consensus_queue.cc:1035] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "064a4f51415d47ab9bc5fdf39943172a" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 03:02:33.408864 8964 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 03:02:35.001554 8148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:35.001837 8148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:35.006258 8148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc4109519b1d48bcbb3475e2cd648670: Sending DeleteTablet for 3 replicas of tablet 024df21e840948f39fdbc18efc3d9947
I20260807 03:02:35.007167 8534 tablet_service.cc:1514] Processing DeleteTablet for tablet 024df21e840948f39fdbc18efc3d9947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:34 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:41628
I20260807 03:02:35.007206 8332 tablet_service.cc:1514] Processing DeleteTablet for tablet 024df21e840948f39fdbc18efc3d9947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:34 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59440
I20260807 03:02:35.007349 8737 tablet_service.cc:1514] Processing DeleteTablet for tablet 024df21e840948f39fdbc18efc3d9947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:34 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59110
I20260807 03:02:35.007957 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 8240
I20260807 03:02:35.020334 9084 tablet_replica.cc:330] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: stopping tablet replica
I20260807 03:02:35.020601 9084 raft_consensus.cc:2229] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:35.020910 9084 raft_consensus.cc:2258] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:35.022146 9084 ts_tablet_manager.cc:1905] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:35.026029 9080 tablet_replica.cc:330] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: stopping tablet replica
I20260807 03:02:35.026304 9080 raft_consensus.cc:2229] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:35.026386 9085 tablet_replica.cc:330] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: stopping tablet replica
I20260807 03:02:35.026589 9085 raft_consensus.cc:2229] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:35.026854 9085 raft_consensus.cc:2258] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:35.026908 9080 raft_consensus.cc:2258] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:35.028148 9085 ts_tablet_manager.cc:1905] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:35.028908 9080 ts_tablet_manager.cc:1905] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:35.031342 9084 ts_tablet_manager.cc:1918] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 03:02:35.031522 9084 log.cc:1196] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/024df21e840948f39fdbc18efc3d9947
I20260807 03:02:35.032419 9084 ts_tablet_manager.cc:1939] T 024df21e840948f39fdbc18efc3d9947 P 064a4f51415d47ab9bc5fdf39943172a: Deleting consensus metadata
I20260807 03:02:35.033720 8130 catalog_manager.cc:4841] TS 064a4f51415d47ab9bc5fdf39943172a (127.5.122.1:33515): tablet 024df21e840948f39fdbc18efc3d9947 (table test-table [id=b89978e50b37413ca67f2a6a70e92e0c]) successfully deleted
I20260807 03:02:35.034775 9085 ts_tablet_manager.cc:1918] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 03:02:35.034987 9085 log.cc:1196] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/024df21e840948f39fdbc18efc3d9947
I20260807 03:02:35.035470 9085 ts_tablet_manager.cc:1939] T 024df21e840948f39fdbc18efc3d9947 P 163f3e6f66504b3e8f5b02dfdf3bdf48: Deleting consensus metadata
I20260807 03:02:35.036567 8130 catalog_manager.cc:4841] TS 163f3e6f66504b3e8f5b02dfdf3bdf48 (127.5.122.3:44445): tablet 024df21e840948f39fdbc18efc3d9947 (table test-table [id=b89978e50b37413ca67f2a6a70e92e0c]) successfully deleted
I20260807 03:02:35.039269 9080 ts_tablet_manager.cc:1918] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 03:02:35.039417 9080 log.cc:1196] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/024df21e840948f39fdbc18efc3d9947
Aug 07 03:02:35 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:35.039795 9080 ts_tablet_manager.cc:1939] T 024df21e840948f39fdbc18efc3d9947 P 831e70d9006f494ba1a6987f7287f9bc: Deleting consensus metadata
I20260807 03:02:35.040843 8130 catalog_manager.cc:4841] TS 831e70d9006f494ba1a6987f7287f9bc (127.5.122.2:34711): tablet 024df21e840948f39fdbc18efc3d9947 (table test-table [id=b89978e50b37413ca67f2a6a70e92e0c]) successfully deleted
I20260807 03:02:35.046514 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:35.009775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.5.122.1:33515)
0807 03:02:35.025032 (+ 15257us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:35.025040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:35.025066 (+ 26us) server_negotiation.cc:407] Connection header received
0807 03:02:35.025128 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:35.025134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:35.025225 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:35.025337 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:35.027326 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:35.028143 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:35.030412 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:35.031158 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:35.031256 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:35.040894 (+ 9638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:35.040913 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:35.040931 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:35.040980 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:35.043844 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:35.044598 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:35.044605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:35.044608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:35.044744 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:35.045051 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:35.045054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:35.045056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:35.045461 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:35.045669 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:35.045994 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 03:02:35.046170 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15094,"thread_start_us":15690,"threads_started":1}
W20260807 03:02:35.047261 8429 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39666
W20260807 03:02:35.047963 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 8240: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:35.048233 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 8240
I20260807 03:02:35.080391 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 8442
Aug 07 03:02:35 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:02:35.094579 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:35.081186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.5.122.2:34711)
0807 03:02:35.081725 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:35.081729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:35.081746 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:02:35.081801 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:35.081805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:35.081878 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:35.081969 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:35.083177 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:35.083928 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:35.085161 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:35.085557 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:35.085642 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:35.089524 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:35.089537 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:35.089551 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:35.089583 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:35.092344 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:35.092910 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:35.092916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:35.092918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:35.093053 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:35.093307 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:35.093310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:35.093312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:35.093633 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:35.093860 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:35.094045 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 03:02:35.094207 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":275,"threads_started":1}
W20260807 03:02:35.095212 8628 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35024
W20260807 03:02:35.095827 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 8442: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:35.095975 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 8442
I20260807 03:02:35.125403 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 8640
Aug 07 03:02:35 dist-test-slave-fgb3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:02:35.140020 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:35.126092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.5.122.3:44445)
0807 03:02:35.126595 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:35.126599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:35.126616 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:02:35.126672 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:35.126676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:35.126739 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:35.126824 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:35.128021 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:35.128740 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:35.129891 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:35.130285 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:35.130428 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:35.134615 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:35.134632 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:35.134649 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:35.134679 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:35.137272 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:35.137885 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:35.137893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:35.137895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:35.138037 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:35.138513 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:35.138518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:35.138522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:35.138871 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:35.139151 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:35.139475 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 03:02:35.139699 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":266,"threads_started":1}
W20260807 03:02:35.140668 8818 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59286
W20260807 03:02:35.141260 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 8640: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:35.141425 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 8640
I20260807 03:02:35.168605 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 8102
I20260807 03:02:35.169817 8202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:35.284159 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 8102
2026-08-07T03:02:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8742 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:35 dist-test-slave-fgb3 krb5kdc[9095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:35 dist-test-slave-fgb3 krb5kdc[9095](info): set up 2 sockets
Aug 07 03:02:35 dist-test-slave-fgb3 krb5kdc[9095](info): commencing operation
krb5kdc: starting...
W20260807 03:02:36.448380 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:36 dist-test-slave-fgb3 krb5kdc[9095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:36.635375 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42771
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:42771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:36.945122 9120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:36.945421 9120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:36.945559 9120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:36.952711 9120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:36.952821 9120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:36.952900 9120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:36.952977 9120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:36.953037 9120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:36.962402 9120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:42771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:36.963639 9120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:36.965042 9120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:36.976508 9125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:36.976549 9126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:36.976574 9128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:36.979553 9120 server_base.cc:1034] running on GCE node
I20260807 03:02:36.980283 9120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:36.981626 9120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:36.982822 9120 hybrid_clock.cc:648] HybridClock initialized: now 1786071756982780 us; error 25 us; skew 500 ppm
I20260807 03:02:36.983139 9120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:36 dist-test-slave-fgb3 krb5kdc[9095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071756, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:36.985960 9120 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:36.987433 9120 webserver.cc:460] Webserver started at http://127.5.122.62:33305/ using document root <none> and password file <none>
I20260807 03:02:36.988107 9120 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:36.988224 9120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:36.988513 9120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:36.991451 9120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "75b3073c88dc4ddfb903f11c7a44c408"
format_stamp: "Formatted at 2026-08-07 03:02:36 on dist-test-slave-fgb3"
server_key: "de28099e7ea967248e6e16472856c854"
server_key_iv: "0b4e298d231e6536dba3d05ed1728420"
server_key_version: "encryptionkey@0"
I20260807 03:02:36.992189 9120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "75b3073c88dc4ddfb903f11c7a44c408"
format_stamp: "Formatted at 2026-08-07 03:02:36 on dist-test-slave-fgb3"
server_key: "de28099e7ea967248e6e16472856c854"
server_key_iv: "0b4e298d231e6536dba3d05ed1728420"
server_key_version: "encryptionkey@0"
I20260807 03:02:36.996940 9120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 03:02:37.000223 9139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:37.001434 9120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:37.001590 9120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "75b3073c88dc4ddfb903f11c7a44c408"
format_stamp: "Formatted at 2026-08-07 03:02:36 on dist-test-slave-fgb3"
server_key: "de28099e7ea967248e6e16472856c854"
server_key_iv: "0b4e298d231e6536dba3d05ed1728420"
server_key_version: "encryptionkey@0"
I20260807 03:02:37.001827 9120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:37.037667 9120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:37.042411 9120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:37.042690 9120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:37.060804 9120 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:42771
I20260807 03:02:37.060878 9207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:42771 every 8 connection(s)
I20260807 03:02:37.062193 9120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:37.067036 9208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:37.070770 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 9120
I20260807 03:02:37.070990 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:37.071646 5608 external_mini_cluster.cc:1428] Setting key f40223b454834d0ea4443c6d027ce27e
I20260807 03:02:37.081058 9208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408: Bootstrap starting.
I20260807 03:02:37.085206 9208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408: Neither blocks nor log segments found. Creating new log.
Aug 07 03:02:37 dist-test-slave-fgb3 krb5kdc[9095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:37.086345 9208 log.cc:824] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:37.089363 9208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408: No bootstrap required, opened a new log
I20260807 03:02:37.094063 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:37.073659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.5.122.62:42771)
0807 03:02:37.074506 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:37.074519 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:37.074560 (+ 41us) server_negotiation.cc:407] Connection header received
0807 03:02:37.076202 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:37.076225 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:37.076608 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:37.076911 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:37.078728 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:37.080022 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:37.081704 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:37.082447 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:37.082580 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:37.087138 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:37.087146 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:37.087159 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:37.087201 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:37.090088 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:37.090826 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:37.090836 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:37.090844 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:37.091028 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:37.091399 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:37.091405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:37.091408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:37.091946 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:37.092258 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:37.092687 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 03:02:37.093084 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":392,"threads_started":1}
I20260807 03:02:37.097376 9208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3073c88dc4ddfb903f11c7a44c408" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42771 } }
I20260807 03:02:37.097719 9208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:37.097815 9208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b3073c88dc4ddfb903f11c7a44c408, State: Initialized, Role: FOLLOWER
I20260807 03:02:37.098367 9208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3073c88dc4ddfb903f11c7a44c408" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42771 } }
I20260807 03:02:37.098621 9208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:37.098742 9208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:37.098883 9208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:37.115641 9208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3073c88dc4ddfb903f11c7a44c408" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42771 } }
I20260807 03:02:37.116138 9208 leader_election.cc:302] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b3073c88dc4ddfb903f11c7a44c408; no voters:
I20260807 03:02:37.116696 9208 leader_election.cc:288] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:37.116854 9215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:37.117843 9215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [term 1 LEADER]: Becoming Leader. State: Replica: 75b3073c88dc4ddfb903f11c7a44c408, State: Running, Role: LEADER
I20260807 03:02:37.118475 9215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3073c88dc4ddfb903f11c7a44c408" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42771 } }
I20260807 03:02:37.119220 9208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:37.121136 9220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b3073c88dc4ddfb903f11c7a44c408. Latest consensus state: current_term: 1 leader_uuid: "75b3073c88dc4ddfb903f11c7a44c408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3073c88dc4ddfb903f11c7a44c408" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42771 } } }
I20260807 03:02:37.121562 9220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:37.121404 9219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75b3073c88dc4ddfb903f11c7a44c408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3073c88dc4ddfb903f11c7a44c408" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42771 } } }
I20260807 03:02:37.121717 9219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:37.124115 9226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:37.129009 9226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:37.137842 9226 catalog_manager.cc:1347] Generated new cluster ID: 65435ed0439b4097b5590dbdf80bf268
I20260807 03:02:37.137941 9226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:37.167642 9226 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:37.169317 9226 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:37.178575 9226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75b3073c88dc4ddfb903f11c7a44c408: Generated new TSK 0
I20260807 03:02:37.179453 9226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
W20260807 03:02:38.023334 5608 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.122.1: real 0.832s user 0.001s sys 0.013s
I20260807 03:02:38.024586 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42771
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:38.343797 9241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:38.344077 9241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:38.344210 9241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:38.351303 9241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:38.351411 9241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:38.351554 9241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:38.360853 9241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:38.362074 9241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:38.363337 9241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:38.375835 9251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:38.375798 9249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:38.375866 9247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:38.376757 9241 server_base.cc:1034] running on GCE node
I20260807 03:02:38.377336 9241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:38.378218 9241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:38.379420 9241 hybrid_clock.cc:648] HybridClock initialized: now 1786071758379376 us; error 39 us; skew 500 ppm
I20260807 03:02:38.379700 9241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:38 dist-test-slave-fgb3 krb5kdc[9095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071758, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:38.382718 9241 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:38.384157 9241 webserver.cc:460] Webserver started at http://127.5.122.1:35723/ using document root <none> and password file <none>
I20260807 03:02:38.384750 9241 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:38.384851 9241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:38.385109 9241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:38.387856 9241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "aebf125fe66a4bf4982e89c823129dcf"
format_stamp: "Formatted at 2026-08-07 03:02:38 on dist-test-slave-fgb3"
server_key: "df7ea3e1ac2f7970d86adffa9bc11a0f"
server_key_iv: "957008e17507f64e4a50b6333d6fe133"
server_key_version: "encryptionkey@0"
I20260807 03:02:38.388576 9241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "aebf125fe66a4bf4982e89c823129dcf"
format_stamp: "Formatted at 2026-08-07 03:02:38 on dist-test-slave-fgb3"
server_key: "df7ea3e1ac2f7970d86adffa9bc11a0f"
server_key_iv: "957008e17507f64e4a50b6333d6fe133"
server_key_version: "encryptionkey@0"
I20260807 03:02:38.393430 9241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 03:02:38.396692 9259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:38.397920 9241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:38.398066 9241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "aebf125fe66a4bf4982e89c823129dcf"
format_stamp: "Formatted at 2026-08-07 03:02:38 on dist-test-slave-fgb3"
server_key: "df7ea3e1ac2f7970d86adffa9bc11a0f"
server_key_iv: "957008e17507f64e4a50b6333d6fe133"
server_key_version: "encryptionkey@0"
I20260807 03:02:38.398231 9241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:38.410209 9241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:38.414381 9241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:38.414647 9241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:38.415580 9241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:38.416913 9241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:38.416985 9241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:38.417096 9241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:38.417146 9241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:38.449453 9241 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:45579
I20260807 03:02:38.449501 9421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:45579 every 8 connection(s)
I20260807 03:02:38.450791 9241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:38.454663 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 9241
I20260807 03:02:38.454879 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:38.455539 5608 external_mini_cluster.cc:1428] Setting key f55489cb8605535af240f5d0b1eb3025
Aug 07 03:02:38 dist-test-slave-fgb3 krb5kdc[9095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071758, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:38.470605 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:38.453349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:52807 (local address 127.5.122.62:42771)
0807 03:02:38.453812 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:38.453817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:38.454745 (+ 928us) server_negotiation.cc:407] Connection header received
0807 03:02:38.456743 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:38.456748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:38.456833 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:38.456924 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:38.458570 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:38.459318 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:38.460983 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:38.461414 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:38.461523 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:38.465141 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:38.465146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:38.465149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:38.465185 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:38.467617 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:38.468406 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:38.468409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:38.468412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:38.468535 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:38.469022 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:38.469025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:38.469027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:38.469271 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:38.469384 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:38.470114 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 03:02:38.470273 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":218,"threads_started":1}
I20260807 03:02:38.471922 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:38.453832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42771 (local address 127.5.122.1:52807)
0807 03:02:38.454593 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:38.454629 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:38.456499 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:38.457169 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:38.457176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:38.457791 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:38.458403 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:38.458423 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:38.459459 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:38.459464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:38.460851 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:38.460864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:38.461638 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:38.461645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:38.461866 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:38.462494 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:38.462524 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:38.464894 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:38.467750 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:38.467758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:38.467772 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:38.468309 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:38.468632 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:38.468636 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:38.468639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:38.468910 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:38.469387 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:38.469394 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:38.469767 (+ 373us) client_negotiation.cc:769] Sending connection context
0807 03:02:38.470051 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 03:02:38.470641 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":287,"threads_started":1}
I20260807 03:02:38.473428 9422 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42771
I20260807 03:02:38.473722 9422 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:38.474442 9422 heartbeater.cc:511] Master 127.5.122.62:42771 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
I20260807 03:02:38.476357 9159 ts_manager.cc:194] Registered new tserver with Master: aebf125fe66a4bf4982e89c823129dcf (127.5.122.1:45579)
I20260807 03:02:38.477747 9159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:52807
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
W20260807 03:02:38.959247 5608 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.122.2: real 0.501s user 0.000s sys 0.013s
I20260807 03:02:38.960052 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42771
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:39.276016 9430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:39.276283 9430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:39.276410 9430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:39.283344 9430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:39.283457 9430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:39.283607 9430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:39.292762 9430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:39.293886 9430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:39.295106 9430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:39.307762 9437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:39.307760 9438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:39.307765 9440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:39.309080 9430 server_base.cc:1034] running on GCE node
I20260807 03:02:39.309612 9430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:39.310500 9430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:39.311669 9430 hybrid_clock.cc:648] HybridClock initialized: now 1786071759311624 us; error 49 us; skew 500 ppm
I20260807 03:02:39.312007 9430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:39 dist-test-slave-fgb3 krb5kdc[9095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071759, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:39.315871 9430 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:39.317842 9430 webserver.cc:460] Webserver started at http://127.5.122.2:38235/ using document root <none> and password file <none>
I20260807 03:02:39.318536 9430 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:39.318651 9430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:39.318972 9430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:39.321691 9430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "d657a5a75fd0485ea69840deb2167ce5"
format_stamp: "Formatted at 2026-08-07 03:02:39 on dist-test-slave-fgb3"
server_key: "82eda12136767a005de1599a88f2c068"
server_key_iv: "66e16e76b58ebf6ce3c2c012425c5601"
server_key_version: "encryptionkey@0"
I20260807 03:02:39.322484 9430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "d657a5a75fd0485ea69840deb2167ce5"
format_stamp: "Formatted at 2026-08-07 03:02:39 on dist-test-slave-fgb3"
server_key: "82eda12136767a005de1599a88f2c068"
server_key_iv: "66e16e76b58ebf6ce3c2c012425c5601"
server_key_version: "encryptionkey@0"
I20260807 03:02:39.327296 9430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 03:02:39.330497 9448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:39.331655 9430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 03:02:39.331804 9430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "d657a5a75fd0485ea69840deb2167ce5"
format_stamp: "Formatted at 2026-08-07 03:02:39 on dist-test-slave-fgb3"
server_key: "82eda12136767a005de1599a88f2c068"
server_key_iv: "66e16e76b58ebf6ce3c2c012425c5601"
server_key_version: "encryptionkey@0"
I20260807 03:02:39.331964 9430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:39.356884 9430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:39.360906 9430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:39.361167 9430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:39.362119 9430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:39.363477 9430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:39.363550 9430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:39.363660 9430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:39.363710 9430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:39.394188 9430 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:44161
I20260807 03:02:39.394238 9621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:44161 every 8 connection(s)
I20260807 03:02:39.395533 9430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:39.399673 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 9430
I20260807 03:02:39.399860 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:39.400261 5608 external_mini_cluster.cc:1428] Setting key a8c78b0b1c5c502a77cb73b0a2d8ea42
Aug 07 03:02:39 dist-test-slave-fgb3 krb5kdc[9095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071759, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:39.415489 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:39.397957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:58703 (local address 127.5.122.62:42771)
0807 03:02:39.398451 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:39.398455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:39.399394 (+ 939us) server_negotiation.cc:407] Connection header received
0807 03:02:39.401355 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:39.401360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:39.401438 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:39.401575 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:39.403141 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.404211 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.405821 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.406422 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.406535 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:39.409889 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:39.409895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:39.409900 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:39.409939 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:39.412565 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:39.413353 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:39.413357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:39.413359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:39.413481 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:39.413895 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:39.413898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:39.413900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:39.414134 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:39.414238 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:39.415066 (+ 828us) server_negotiation.cc:299] Negotiation successful
0807 03:02:39.415240 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":216,"threads_started":1}
I20260807 03:02:39.416775 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:39.398499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42771 (local address 127.5.122.2:58703)
0807 03:02:39.399214 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:39.399262 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:39.401025 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:39.401748 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:39.401756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:39.402358 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:39.402980 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:39.402998 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.404349 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.404353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:39.405693 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:39.405705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.406609 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.406614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:39.406816 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:39.407436 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:39.407457 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:39.409670 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:39.412702 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:39.412711 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:39.412725 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:39.413253 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:39.413575 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:39.413579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:39.413582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:39.413812 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:39.414279 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:39.414288 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:39.414699 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 03:02:39.414980 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 03:02:39.415474 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":293,"threads_started":1}
I20260807 03:02:39.418172 9622 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42771
I20260807 03:02:39.418484 9622 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:39.419237 9622 heartbeater.cc:511] Master 127.5.122.62:42771 requested a full tablet report, sending...
I20260807 03:02:39.420635 9159 ts_manager.cc:194] Registered new tserver with Master: d657a5a75fd0485ea69840deb2167ce5 (127.5.122.2:44161)
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
I20260807 03:02:39.421437 9159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:58703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:39.447063 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42771
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:02:39.480450 9422 heartbeater.cc:503] Master 127.5.122.62:42771 was elected leader, sending a full tablet report...
W20260807 03:02:39.763741 9628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:39.764043 9628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:39.764182 9628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:39.771976 9628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:39.772111 9628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:39.772305 9628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:39.782622 9628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:39.783924 9628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:39.785271 9628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:39.798149 9637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:39.798206 9635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:39.798236 9640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:39.799259 9628 server_base.cc:1034] running on GCE node
I20260807 03:02:39.799880 9628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:39.800758 9628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:39.801955 9628 hybrid_clock.cc:648] HybridClock initialized: now 1786071759801897 us; error 48 us; skew 500 ppm
I20260807 03:02:39.802289 9628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:39 dist-test-slave-fgb3 krb5kdc[9095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071759, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:39.805527 9628 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:39.807070 9628 webserver.cc:460] Webserver started at http://127.5.122.3:42405/ using document root <none> and password file <none>
I20260807 03:02:39.807760 9628 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:39.807888 9628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:39.808210 9628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:39.811216 9628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "8e330f05061b4e0488c68c9ff6ec689b"
format_stamp: "Formatted at 2026-08-07 03:02:39 on dist-test-slave-fgb3"
server_key: "0d37c39a7cc28e183d634dacb0eeba62"
server_key_iv: "6ca8d4686ccce9693ce8bffb3ea90fd6"
server_key_version: "encryptionkey@0"
I20260807 03:02:39.812038 9628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "8e330f05061b4e0488c68c9ff6ec689b"
format_stamp: "Formatted at 2026-08-07 03:02:39 on dist-test-slave-fgb3"
server_key: "0d37c39a7cc28e183d634dacb0eeba62"
server_key_iv: "6ca8d4686ccce9693ce8bffb3ea90fd6"
server_key_version: "encryptionkey@0"
I20260807 03:02:39.817222 9628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:02:39.820863 9650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:39.822129 9628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:02:39.822305 9628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "8e330f05061b4e0488c68c9ff6ec689b"
format_stamp: "Formatted at 2026-08-07 03:02:39 on dist-test-slave-fgb3"
server_key: "0d37c39a7cc28e183d634dacb0eeba62"
server_key_iv: "6ca8d4686ccce9693ce8bffb3ea90fd6"
server_key_version: "encryptionkey@0"
I20260807 03:02:39.822532 9628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:39.846671 9628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:39.851127 9628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:39.851439 9628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:39.852478 9628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:39.853916 9628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:39.854004 9628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:39.854138 9628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:39.854200 9628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:39.885645 9628 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:45743
I20260807 03:02:39.885690 9809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:45743 every 8 connection(s)
I20260807 03:02:39.886942 9628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:39.896219 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 9628
I20260807 03:02:39.896396 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:39.896842 5608 external_mini_cluster.cc:1428] Setting key 271de9b056e8a432174967869ac49048
Aug 07 03:02:39 dist-test-slave-fgb3 krb5kdc[9095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071759, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:39.905468 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:39.889323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:55877 (local address 127.5.122.62:42771)
0807 03:02:39.889539 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:39.889545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:39.890596 (+ 1051us) server_negotiation.cc:407] Connection header received
0807 03:02:39.892322 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:39.892326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:39.892396 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:39.892481 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:39.893836 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.894615 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.896061 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.896582 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.896700 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:39.900418 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:39.900423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:39.900426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:39.900457 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:39.902927 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:39.903609 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:39.903612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:39.903614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:39.903728 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:39.904065 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:39.904068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:39.904070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:39.904329 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:39.904433 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:39.905059 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 03:02:39.905224 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 03:02:39.906494 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:39.889725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42771 (local address 127.5.122.3:55877)
0807 03:02:39.890448 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:39.890488 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:39.892092 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:39.892653 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:39.892660 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:39.893139 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:39.893709 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:39.893726 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.894734 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.894738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:39.895919 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:39.895931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:39.896815 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:39.896823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:39.897074 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:39.897668 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:39.897689 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:39.900239 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:39.903054 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:39.903063 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:39.903078 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:39.903519 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:39.903801 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:39.903804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:39.903806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:39.903982 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:39.904422 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:39.904427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:39.904734 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 03:02:39.904985 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 03:02:39.905366 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":265,"threads_started":1}
I20260807 03:02:39.907795 9812 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42771
I20260807 03:02:39.908075 9812 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:39.908731 9812 heartbeater.cc:511] Master 127.5.122.62:42771 requested a full tablet report, sending...
I20260807 03:02:39.910059 9159 ts_manager.cc:194] Registered new tserver with Master: 8e330f05061b4e0488c68c9ff6ec689b (127.5.122.3:45743)
I20260807 03:02:39.910890 9159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:55877
I20260807 03:02:39.911944 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 03:02:39.945098 9624 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:02:39.942660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.5.122.62:42771)
0807 03:02:39.942870 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:39.942878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:39.943286 (+ 408us) server_negotiation.cc:407] Connection header received
0807 03:02:39.943465 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:39.943469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:39.943547 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:39.943624 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:39.944783 (+ 1159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39212: BlockingRecv error: recv got EOF from 127.0.0.1:39212 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 03:02:39.948583 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9241
W20260807 03:02:39.951416 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9241: Not authorized: Client connection negotiation failed: client connection to 127.5.122.1:45579: server requires authentication, but client does not have Kerberos credentials available
I20260807 03:02:39.951598 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 9241
I20260807 03:02:39.973246 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9430
W20260807 03:02:39.975903 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9430: Not authorized: Client connection negotiation failed: client connection to 127.5.122.2:44161: server requires authentication, but client does not have Kerberos credentials available
I20260807 03:02:39.976073 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 9430
I20260807 03:02:39.998612 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9628
W20260807 03:02:40.001323 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9628: Not authorized: Client connection negotiation failed: client connection to 127.5.122.3:45743: server requires authentication, but client does not have Kerberos credentials available
I20260807 03:02:40.001452 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 9628
I20260807 03:02:40.021210 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9120
I20260807 03:02:40.022528 9203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:40.126981 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 9120
2026-08-07T03:02:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4837 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:40 dist-test-slave-fgb3 krb5kdc[9833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:40 dist-test-slave-fgb3 krb5kdc[9833](info): set up 2 sockets
Aug 07 03:02:40 dist-test-slave-fgb3 krb5kdc[9833](info): commencing operation
krb5kdc: starting...
W20260807 03:02:41.296015 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:41 dist-test-slave-fgb3 krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:41.493834 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39563
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:41.806432 9855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:41.806741 9855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:41.806875 9855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:41.814196 9855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:41.814339 9855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:41.814429 9855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:41.814509 9855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:41.814574 9855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:41.824241 9855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:41.825482 9855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:41.826809 9855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:41.837759 9864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:41.837809 9866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:41.837831 9861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:41.838564 9855 server_base.cc:1034] running on GCE node
I20260807 03:02:41.839280 9855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:41.840641 9855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:41.841825 9855 hybrid_clock.cc:648] HybridClock initialized: now 1786071761841779 us; error 40 us; skew 500 ppm
I20260807 03:02:41.842147 9855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:41 dist-test-slave-fgb3 krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071761, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:41.845150 9855 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:41.846556 9855 webserver.cc:460] Webserver started at http://127.5.122.62:42515/ using document root <none> and password file <none>
I20260807 03:02:41.847200 9855 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:41.847318 9855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:41.847600 9855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:41.850494 9855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "00fecc29ab18445da9332118643386dc"
format_stamp: "Formatted at 2026-08-07 03:02:41 on dist-test-slave-fgb3"
server_key: "b4a1d180000963809ee5c4d42fd8a54c"
server_key_iv: "27ab78e0351018921cbf8bd0ec6411f6"
server_key_version: "encryptionkey@0"
I20260807 03:02:41.851254 9855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "00fecc29ab18445da9332118643386dc"
format_stamp: "Formatted at 2026-08-07 03:02:41 on dist-test-slave-fgb3"
server_key: "b4a1d180000963809ee5c4d42fd8a54c"
server_key_iv: "27ab78e0351018921cbf8bd0ec6411f6"
server_key_version: "encryptionkey@0"
I20260807 03:02:41.856202 9855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:02:41.859480 9879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:41.860750 9855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:41.860924 9855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "00fecc29ab18445da9332118643386dc"
format_stamp: "Formatted at 2026-08-07 03:02:41 on dist-test-slave-fgb3"
server_key: "b4a1d180000963809ee5c4d42fd8a54c"
server_key_iv: "27ab78e0351018921cbf8bd0ec6411f6"
server_key_version: "encryptionkey@0"
I20260807 03:02:41.861089 9855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:41.886035 9855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:41.890043 9855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:41.890298 9855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:41.906905 9855 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:39563
I20260807 03:02:41.906940 9950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:39563 every 8 connection(s)
I20260807 03:02:41.908226 9855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:41.911988 9951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:41.911964 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 9855
I20260807 03:02:41.912173 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:41.912982 5608 external_mini_cluster.cc:1428] Setting key 9e8bfbaa2a2349aab4cfeefe05f28f66
I20260807 03:02:41.925876 9951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc: Bootstrap starting.
Aug 07 03:02:41 dist-test-slave-fgb3 krb5kdc[9833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:41.929306 9951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:41.930613 9951 log.cc:824] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:41.933653 9951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc: No bootstrap required, opened a new log
I20260807 03:02:41.935688 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:41.914772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.5.122.62:39563)
0807 03:02:41.915523 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:41.915534 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:41.915574 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:02:41.917037 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:41.917056 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:41.917409 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:41.917725 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:41.919544 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:41.920755 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:41.922510 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:41.923306 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:41.923481 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:41.927794 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:41.927804 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:41.927820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:41.927872 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:41.931492 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:41.932215 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:41.932223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:41.932230 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:41.932429 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:41.932760 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:41.932765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:41.932768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:41.933407 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:41.933694 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:41.934047 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 03:02:41.934477 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":330,"threads_started":1}
I20260807 03:02:41.939644 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:02:41.943482 9951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fecc29ab18445da9332118643386dc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39563 } }
I20260807 03:02:41.943881 9951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:41.943972 9951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00fecc29ab18445da9332118643386dc, State: Initialized, Role: FOLLOWER
I20260807 03:02:41.944538 9951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fecc29ab18445da9332118643386dc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39563 } }
I20260807 03:02:41.944778 9951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:41.944902 9951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:41.945063 9951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:41.947782 9951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fecc29ab18445da9332118643386dc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39563 } }
I20260807 03:02:41.948324 9951 leader_election.cc:302] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00fecc29ab18445da9332118643386dc; no voters:
I20260807 03:02:41.948905 9951 leader_election.cc:288] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:41.948992 9957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:41.949958 9957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [term 1 LEADER]: Becoming Leader. State: Replica: 00fecc29ab18445da9332118643386dc, State: Running, Role: LEADER
I20260807 03:02:41.950533 9957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fecc29ab18445da9332118643386dc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39563 } }
I20260807 03:02:41.951009 9951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:41.953056 9963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00fecc29ab18445da9332118643386dc. Latest consensus state: current_term: 1 leader_uuid: "00fecc29ab18445da9332118643386dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fecc29ab18445da9332118643386dc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39563 } } }
I20260807 03:02:41.953135 9959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00fecc29ab18445da9332118643386dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fecc29ab18445da9332118643386dc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39563 } } }
I20260807 03:02:41.953474 9959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:41.953570 9963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:41.954248 9968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:41.963231 9968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:41.987793 9968 catalog_manager.cc:1347] Generated new cluster ID: 9146e6d16c984fa3b842534134ec2224
I20260807 03:02:41.987920 9968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:42.012794 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:41.983461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.5.122.62:39563)
0807 03:02:41.983655 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:41.983661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:42.003049 (+ 19388us) server_negotiation.cc:407] Connection header received
0807 03:02:42.003249 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:42.003257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:42.003362 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:42.003496 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:42.005066 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:42.006269 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:42.008321 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:42.009041 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:42.009143 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:42.010703 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:42.010711 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:42.010717 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:42.011808 (+ 1091us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:42.011900 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:42.011931 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:42.012031 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:42.012280 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 03:02:42.012392 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:02:42.022383 9968 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:42.023623 9968 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:42.030941 9968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00fecc29ab18445da9332118643386dc: Generated new TSK 0
I20260807 03:02:42.031688 9968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:02:42.041589 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 9855
I20260807 03:02:42.042898 9944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:42.136536 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 9855
2026-08-07T03:02:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2003 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:42 dist-test-slave-fgb3 krb5kdc[9999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:42 dist-test-slave-fgb3 krb5kdc[9999](info): set up 2 sockets
Aug 07 03:02:42 dist-test-slave-fgb3 krb5kdc[9999](info): commencing operation
krb5kdc: starting...
W20260807 03:02:43.303218 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:43 dist-test-slave-fgb3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:43.491545 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45065
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:36203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:45065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:43.795349 10018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:43.795634 10018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:43.795769 10018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:43.802762 10018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:43.802870 10018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:43.802944 10018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:43.803018 10018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:43.803076 10018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:43.812479 10018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:36203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:45065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:43.813696 10018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:43.814985 10018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:43.825711 10024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:43.825806 10025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:43.825775 10029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:43.826763 10018 server_base.cc:1034] running on GCE node
I20260807 03:02:43.827466 10018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:43.828737 10018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:43.829937 10018 hybrid_clock.cc:648] HybridClock initialized: now 1786071763829898 us; error 27 us; skew 500 ppm
I20260807 03:02:43.830250 10018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:43 dist-test-slave-fgb3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071763, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:43.833081 10018 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:43.834508 10018 webserver.cc:460] Webserver started at http://127.5.122.62:34771/ using document root <none> and password file <none>
I20260807 03:02:43.835114 10018 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:43.835230 10018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:43.835515 10018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:43.838347 10018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "33e6d9f1d7914c3582a8d2f19218a541"
format_stamp: "Formatted at 2026-08-07 03:02:43 on dist-test-slave-fgb3"
server_key: "2a411330eaf817cddcd7d81a4fc0ee97"
server_key_iv: "b777f023fc106d72cf1d1958313d1c61"
server_key_version: "encryptionkey@0"
I20260807 03:02:43.839231 10018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "33e6d9f1d7914c3582a8d2f19218a541"
format_stamp: "Formatted at 2026-08-07 03:02:43 on dist-test-slave-fgb3"
server_key: "2a411330eaf817cddcd7d81a4fc0ee97"
server_key_iv: "b777f023fc106d72cf1d1958313d1c61"
server_key_version: "encryptionkey@0"
I20260807 03:02:43.844056 10018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:02:43.847204 10039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:43.848402 10018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:02:43.848552 10018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "33e6d9f1d7914c3582a8d2f19218a541"
format_stamp: "Formatted at 2026-08-07 03:02:43 on dist-test-slave-fgb3"
server_key: "2a411330eaf817cddcd7d81a4fc0ee97"
server_key_iv: "b777f023fc106d72cf1d1958313d1c61"
server_key_version: "encryptionkey@0"
I20260807 03:02:43.848717 10018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:43.867344 10018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:43.870854 10018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:43.871095 10018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:43.887712 10018 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:45065
I20260807 03:02:43.887748 10108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:45065 every 8 connection(s)
I20260807 03:02:43.888921 10018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:43.892325 10109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:43.897068 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 10018
I20260807 03:02:43.897282 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:43.898084 5608 external_mini_cluster.cc:1428] Setting key 006b391ac0d23de7f6fdf23065eac4bd
I20260807 03:02:43.903334 10109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541: Bootstrap starting.
I20260807 03:02:43.906832 10109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:43.908039 10109 log.cc:824] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:43.911391 10109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541: No bootstrap required, opened a new log
Aug 07 03:02:43 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:43.919333 10109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e6d9f1d7914c3582a8d2f19218a541" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45065 } }
I20260807 03:02:43.919747 10109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:43.919842 10109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e6d9f1d7914c3582a8d2f19218a541, State: Initialized, Role: FOLLOWER
I20260807 03:02:43.920387 10109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e6d9f1d7914c3582a8d2f19218a541" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45065 } }
I20260807 03:02:43.920595 10109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:43.920733 10109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:43.920889 10109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:43.920989 10116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:43.900126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.5.122.62:45065)
0807 03:02:43.900877 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:43.900888 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:43.900932 (+ 44us) server_negotiation.cc:407] Connection header received
0807 03:02:43.902653 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:43.902675 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:43.903050 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:43.903356 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:43.905007 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:43.906354 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:43.908069 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:43.908990 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:43.909189 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:43.913619 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:43.913627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:43.913643 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:43.913684 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:43.916860 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:43.917481 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:43.917490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:43.917496 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:43.917671 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:43.918049 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:43.918053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:43.918055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:43.918640 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:43.918916 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:43.919366 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 03:02:43.919798 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":336,"threads_started":1}
I20260807 03:02:43.923293 10109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e6d9f1d7914c3582a8d2f19218a541" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45065 } }
I20260807 03:02:43.923856 10109 leader_election.cc:302] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33e6d9f1d7914c3582a8d2f19218a541; no voters:
I20260807 03:02:43.924422 10109 leader_election.cc:288] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:43.924533 10118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:43.925174 10118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [term 1 LEADER]: Becoming Leader. State: Replica: 33e6d9f1d7914c3582a8d2f19218a541, State: Running, Role: LEADER
I20260807 03:02:43.925685 10118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e6d9f1d7914c3582a8d2f19218a541" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45065 } }
I20260807 03:02:43.926252 10109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:43.928547 10120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e6d9f1d7914c3582a8d2f19218a541. Latest consensus state: current_term: 1 leader_uuid: "33e6d9f1d7914c3582a8d2f19218a541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e6d9f1d7914c3582a8d2f19218a541" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45065 } } }
I20260807 03:02:43.928617 10119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33e6d9f1d7914c3582a8d2f19218a541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e6d9f1d7914c3582a8d2f19218a541" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45065 } } }
I20260807 03:02:43.928900 10120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:43.928970 10119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:43.930281 10127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:43.934504 10127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:43.942028 10127 catalog_manager.cc:1347] Generated new cluster ID: c49f9db4ea5e4c32a3153c44494ecbe9
I20260807 03:02:43.942137 10127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:43.957190 10127 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:43.958930 10127 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:43.966846 10127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33e6d9f1d7914c3582a8d2f19218a541: Generated new TSK 0
I20260807 03:02:43.967757 10127 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:02:44.023237 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45065
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:36203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:44.340065 10149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:44.340349 10149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:44.340480 10149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:44.347645 10149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:44.347756 10149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:44.347910 10149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:44.357659 10149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:36203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:44.358968 10149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:44.360239 10149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:44.372438 10159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:44.372530 10155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:44.372550 10154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:44.373203 10149 server_base.cc:1034] running on GCE node
I20260807 03:02:44.373777 10149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:44.374719 10149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:44.375918 10149 hybrid_clock.cc:648] HybridClock initialized: now 1786071764375868 us; error 44 us; skew 500 ppm
I20260807 03:02:44.376202 10149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:44 dist-test-slave-fgb3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071764, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:44.379248 10149 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:44.380530 10149 webserver.cc:460] Webserver started at http://127.5.122.1:32807/ using document root <none> and password file <none>
I20260807 03:02:44.381156 10149 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:44.381268 10149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:44.381546 10149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:44.384315 10149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "14f4a54582db4f2d8829ec32e784df36"
format_stamp: "Formatted at 2026-08-07 03:02:44 on dist-test-slave-fgb3"
server_key: "ff1bb91ff99c3729d71954529978c2ad"
server_key_iv: "976e534cd103fae99b2140355de04ac2"
server_key_version: "encryptionkey@0"
I20260807 03:02:44.385072 10149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "14f4a54582db4f2d8829ec32e784df36"
format_stamp: "Formatted at 2026-08-07 03:02:44 on dist-test-slave-fgb3"
server_key: "ff1bb91ff99c3729d71954529978c2ad"
server_key_iv: "976e534cd103fae99b2140355de04ac2"
server_key_version: "encryptionkey@0"
I20260807 03:02:44.389925 10149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 03:02:44.393172 10168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:44.394415 10149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:02:44.394554 10149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "14f4a54582db4f2d8829ec32e784df36"
format_stamp: "Formatted at 2026-08-07 03:02:44 on dist-test-slave-fgb3"
server_key: "ff1bb91ff99c3729d71954529978c2ad"
server_key_iv: "976e534cd103fae99b2140355de04ac2"
server_key_version: "encryptionkey@0"
I20260807 03:02:44.394718 10149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:44.426471 10149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:44.430243 10149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:44.430526 10149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:44.431413 10149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:44.432736 10149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:44.432811 10149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:44.432934 10149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:44.432986 10149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:44.462543 10149 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:44911
I20260807 03:02:44.462579 10329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:44911 every 8 connection(s)
I20260807 03:02:44.463734 10149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:44.471594 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 10149
I20260807 03:02:44.471786 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:44.472415 5608 external_mini_cluster.cc:1428] Setting key d5319335d3b61d03fd337e78b352e887
Aug 07 03:02:44 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071764, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:44.482648 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:44.466022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:42289 (local address 127.5.122.62:45065)
0807 03:02:44.466554 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:44.466560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:44.467358 (+ 798us) server_negotiation.cc:407] Connection header received
0807 03:02:44.469045 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:44.469050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:44.469118 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:44.469235 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:44.470720 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.471587 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.473117 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.473646 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.473764 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:44.476978 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:44.476985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:44.476988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:44.477024 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:44.479865 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:44.480507 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:44.480512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:44.480514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:44.480654 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:44.481002 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:44.481007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:44.481009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:44.481309 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:44.481465 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:44.482045 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 03:02:44.482257 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":219,"threads_started":1}
I20260807 03:02:44.483352 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:44.466488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45065 (local address 127.5.122.1:42289)
0807 03:02:44.467208 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:44.467245 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:44.468796 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:44.469442 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:44.469450 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:44.469988 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:44.470554 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:44.470574 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.471728 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.471732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:44.472977 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:44.472989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.473846 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.473851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:44.474094 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:44.474739 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:44.474763 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:44.476782 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:44.479986 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:44.479992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:44.480002 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:44.480391 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:44.480739 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:44.480741 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:44.480744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:44.480903 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:44.481429 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:44.481434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:44.481702 (+ 268us) client_negotiation.cc:769] Sending connection context
0807 03:02:44.481907 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 03:02:44.482237 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":257,"threads_started":1}
I20260807 03:02:44.484959 10330 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45065
I20260807 03:02:44.485248 10330 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:44.485971 10330 heartbeater.cc:511] Master 127.5.122.62:45065 requested a full tablet report, sending...
I20260807 03:02:44.488065 10062 ts_manager.cc:194] Registered new tserver with Master: 14f4a54582db4f2d8829ec32e784df36 (127.5.122.1:44911)
I20260807 03:02:44.489568 10062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:42289
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:02:44.517468 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45065
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:36203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:44.825217 10340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:44.825493 10340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:44.825634 10340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:44.832795 10340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:44.832895 10340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:44.833047 10340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:44.842561 10340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:36203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:44.843835 10340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:44.845091 10340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:44.857375 10347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:44.857409 10348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:44.857343 10352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:44.858574 10340 server_base.cc:1034] running on GCE node
I20260807 03:02:44.859119 10340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:44.860006 10340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:44.861270 10340 hybrid_clock.cc:648] HybridClock initialized: now 1786071764861203 us; error 26 us; skew 500 ppm
I20260807 03:02:44.861608 10340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:44 dist-test-slave-fgb3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071764, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:44.864754 10340 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:44.866209 10340 webserver.cc:460] Webserver started at http://127.5.122.2:39281/ using document root <none> and password file <none>
I20260807 03:02:44.867049 10340 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:44.867184 10340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:44.867530 10340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:44.870596 10340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "74e8a93c908840578032f37182df36ba"
format_stamp: "Formatted at 2026-08-07 03:02:44 on dist-test-slave-fgb3"
server_key: "792b21eb79009427b6af201f184accb2"
server_key_iv: "502c3e1f3948e8f2fd42c05f68febc68"
server_key_version: "encryptionkey@0"
I20260807 03:02:44.871486 10340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "74e8a93c908840578032f37182df36ba"
format_stamp: "Formatted at 2026-08-07 03:02:44 on dist-test-slave-fgb3"
server_key: "792b21eb79009427b6af201f184accb2"
server_key_iv: "502c3e1f3948e8f2fd42c05f68febc68"
server_key_version: "encryptionkey@0"
I20260807 03:02:44.876749 10340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:02:44.880090 10363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:44.881357 10340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 03:02:44.881507 10340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "74e8a93c908840578032f37182df36ba"
format_stamp: "Formatted at 2026-08-07 03:02:44 on dist-test-slave-fgb3"
server_key: "792b21eb79009427b6af201f184accb2"
server_key_iv: "502c3e1f3948e8f2fd42c05f68febc68"
server_key_version: "encryptionkey@0"
I20260807 03:02:44.881680 10340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:44.906855 10340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:44.911159 10340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:44.911420 10340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:44.912322 10340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:44.913620 10340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:44.913697 10340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:44.913793 10340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:44.913836 10340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:44.944634 10340 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:43273
I20260807 03:02:44.944716 10526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:43273 every 8 connection(s)
I20260807 03:02:44.945906 10340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:44.955814 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 10340
I20260807 03:02:44.955963 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:44.956418 5608 external_mini_cluster.cc:1428] Setting key 53010bc1532abe0d9c850a353260e698
Aug 07 03:02:44 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071764, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:44.965036 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:44.948408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:41717 (local address 127.5.122.62:45065)
0807 03:02:44.948612 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:44.948618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:44.949806 (+ 1188us) server_negotiation.cc:407] Connection header received
0807 03:02:44.951610 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:44.951615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:44.951706 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:44.951795 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:44.953354 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.954085 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.955651 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.956176 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.956286 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:44.959633 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:44.959639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:44.959642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:44.959673 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:44.962064 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:44.962809 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:44.962812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:44.962815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:44.962940 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:44.963389 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:44.963393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:44.963395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:44.963658 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:44.963813 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:44.964466 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 03:02:44.964690 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:02:44.965962 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:44.948913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45065 (local address 127.5.122.2:41717)
0807 03:02:44.949642 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:44.949679 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:44.951352 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:44.952028 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:44.952036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:44.952581 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:44.953157 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:44.953175 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.954235 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.954238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:44.955516 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:44.955528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:44.956412 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:44.956421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:44.956657 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:44.957304 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:44.957333 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:44.959425 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:44.962188 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:44.962198 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:44.962214 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:44.962684 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:44.963041 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:44.963046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:44.963050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:44.963230 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:44.963786 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:44.963793 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:44.964114 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 03:02:44.964349 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 03:02:44.964856 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":289,"threads_started":1}
I20260807 03:02:44.967378 10528 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45065
I20260807 03:02:44.967672 10528 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:44.968398 10528 heartbeater.cc:511] Master 127.5.122.62:45065 requested a full tablet report, sending...
I20260807 03:02:44.969832 10062 ts_manager.cc:194] Registered new tserver with Master: 74e8a93c908840578032f37182df36ba (127.5.122.2:43273)
I20260807 03:02:44.970875 10062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:41717
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:45.005143 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45065
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:36203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:45.324065 10535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:45.324360 10535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:45.324468 10535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:45.331849 10535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:45.331964 10535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:45.332087 10535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:45.341917 10535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:36203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:45.343228 10535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:45.344542 10535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:45.357177 10547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:45.357287 10548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:45.357504 10550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:45.358130 10535 server_base.cc:1034] running on GCE node
I20260807 03:02:45.358747 10535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:45.359664 10535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:45.360855 10535 hybrid_clock.cc:648] HybridClock initialized: now 1786071765360809 us; error 45 us; skew 500 ppm
I20260807 03:02:45.361166 10535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071765, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:45.366860 10535 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:45.368288 10535 webserver.cc:460] Webserver started at http://127.5.122.3:36037/ using document root <none> and password file <none>
I20260807 03:02:45.368963 10535 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:45.369079 10535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:45.369382 10535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:45.372257 10535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "83d3cd193ac14d44b248ba4ccdf9a0d6"
format_stamp: "Formatted at 2026-08-07 03:02:45 on dist-test-slave-fgb3"
server_key: "366d62fa518d0ec66fddf6429a35d279"
server_key_iv: "2a294a1e42916cb256223bdc46304f91"
server_key_version: "encryptionkey@0"
I20260807 03:02:45.373026 10535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "83d3cd193ac14d44b248ba4ccdf9a0d6"
format_stamp: "Formatted at 2026-08-07 03:02:45 on dist-test-slave-fgb3"
server_key: "366d62fa518d0ec66fddf6429a35d279"
server_key_iv: "2a294a1e42916cb256223bdc46304f91"
server_key_version: "encryptionkey@0"
I20260807 03:02:45.378118 10535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 03:02:45.381584 10561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:45.382975 10535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:02:45.383127 10535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "83d3cd193ac14d44b248ba4ccdf9a0d6"
format_stamp: "Formatted at 2026-08-07 03:02:45 on dist-test-slave-fgb3"
server_key: "366d62fa518d0ec66fddf6429a35d279"
server_key_iv: "2a294a1e42916cb256223bdc46304f91"
server_key_version: "encryptionkey@0"
I20260807 03:02:45.383297 10535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:45.396853 10535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:45.400849 10535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:45.401108 10535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:45.401988 10535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:45.403271 10535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:45.403347 10535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:45.403474 10535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:45.403542 10535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:45.434432 10535 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:41199
I20260807 03:02:45.434487 10730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:41199 every 8 connection(s)
I20260807 03:02:45.435638 10535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:45.445712 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 10535
I20260807 03:02:45.445883 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:45.446396 5608 external_mini_cluster.cc:1428] Setting key 1c4748d07ba724ec45f7dc68b01ff853
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071765, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:45.454463 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:45.438079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:56949 (local address 127.5.122.62:45065)
0807 03:02:45.438254 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:45.438260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:45.439400 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 03:02:45.441090 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:45.441094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:45.441194 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:45.441282 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:45.442667 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.443375 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.444844 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.445225 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.445310 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:45.448917 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:45.448922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:45.448925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:45.448957 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:45.451610 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.452273 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.452277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.452280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.452442 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.452842 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.452847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.452850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.453193 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:45.453327 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:45.453972 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 03:02:45.454139 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:02:45.455312 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:45.438553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45065 (local address 127.5.122.3:56949)
0807 03:02:45.439238 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:45.439278 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:45.440858 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:45.441470 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:45.441477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:45.441961 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:45.442527 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:45.442546 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.443524 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.443528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:45.444729 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:45.444742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.445401 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.445405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:45.445590 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:45.446231 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:45.446253 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:45.448703 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:45.451739 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:45.451745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:45.451755 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:45.452147 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:45.452530 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:45.452533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:45.452536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:45.452734 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:45.453348 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:45.453356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:45.453659 (+ 303us) client_negotiation.cc:769] Sending connection context
0807 03:02:45.453890 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 03:02:45.454250 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":267,"threads_started":1}
I20260807 03:02:45.456640 10731 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45065
I20260807 03:02:45.456918 10731 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:45.457590 10731 heartbeater.cc:511] Master 127.5.122.62:45065 requested a full tablet report, sending...
I20260807 03:02:45.458937 10062 ts_manager.cc:194] Registered new tserver with Master: 83d3cd193ac14d44b248ba4ccdf9a0d6 (127.5.122.3:41199)
I20260807 03:02:45.459807 10062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:56949
I20260807 03:02:45.461587 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071765, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
I20260807 03:02:45.492447 10330 heartbeater.cc:503] Master 127.5.122.62:45065 was elected leader, sending a full tablet report...
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071765, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:45.511234 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:45.498337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.5.122.62:45065)
0807 03:02:45.498544 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:45.498551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:45.498999 (+ 448us) server_negotiation.cc:407] Connection header received
0807 03:02:45.499178 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:45.499184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:45.499282 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:45.499392 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:45.500783 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.501496 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.502698 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.503088 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.503173 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:45.506715 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:45.506720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:45.506724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:45.506757 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:45.509206 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.509722 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.509725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.509728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.509856 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.510128 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.510131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.510133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.510483 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:45.510671 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:45.510862 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 03:02:45.511014 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 03:02:45.511757 10062 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58296
I20260807 03:02:45.512955 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10149
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071765, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:45.528343 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:45.514305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.5.122.1:44911)
0807 03:02:45.514962 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:45.514969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:45.514992 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:02:45.515044 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:45.515050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:45.515251 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:45.515386 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:45.516508 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.517516 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.518747 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.519370 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.519494 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:45.522997 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:45.523008 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:45.523020 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:45.523050 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:45.525621 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.526306 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.526341 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.526344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.526477 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.526832 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.526837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.526840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.527164 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:45.527468 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:45.527752 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 03:02:45.528013 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":295,"threads_started":1}
W20260807 03:02:45.529585 10324 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33230
W20260807 03:02:45.530606 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10149: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:45.530793 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 10149
I20260807 03:02:45.551537 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10340
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071765, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:02:45.566205 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:45.552456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.5.122.2:43273)
0807 03:02:45.553012 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:45.553019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:45.553045 (+ 26us) server_negotiation.cc:407] Connection header received
0807 03:02:45.553103 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:45.553110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:45.553324 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:45.553450 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:45.554682 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.555705 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.556871 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.557430 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.557524 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:45.561261 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:45.561289 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:45.561302 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:45.561343 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:45.564030 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.564585 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.564591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.564593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.564713 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.564988 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.564991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.564993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.565329 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:45.565566 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:45.565719 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 03:02:45.565915 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":274,"threads_started":1}
W20260807 03:02:45.567468 10521 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45322
W20260807 03:02:45.568523 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10340: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:45.568655 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 10340
I20260807 03:02:45.588238 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10535
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071765, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:02:45.603106 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:45.589068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35112 (local address 127.5.122.3:41199)
0807 03:02:45.589643 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:45.589650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:45.589678 (+ 28us) server_negotiation.cc:407] Connection header received
0807 03:02:45.589736 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:45.589742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:45.589955 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:45.590080 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:45.591416 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.592465 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.593621 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:45.594164 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:45.594255 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:45.598265 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:45.598277 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:45.598290 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:45.598349 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:45.600754 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.601302 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.601308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.601310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.601418 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:45.601704 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:45.601707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:45.601709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:45.602023 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:45.602245 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:45.602556 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 03:02:45.602752 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
W20260807 03:02:45.604249 10723 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35112
W20260807 03:02:45.605226 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10535: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:02:45.605376 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 10535
I20260807 03:02:45.625250 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10018
I20260807 03:02:45.626407 10104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:45.729707 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 10018
2026-08-07T03:02:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3598 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[10756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[10756](info): set up 2 sockets
Aug 07 03:02:45 dist-test-slave-fgb3 krb5kdc[10756](info): commencing operation
krb5kdc: starting...
W20260807 03:02:46.908411 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:47 dist-test-slave-fgb3 krb5kdc[10756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:47.106921 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:37497
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:37497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:47.428553 10777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:47.428829 10777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:47.428958 10777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:47.435843 10777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:47.435950 10777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:47.436034 10777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:47.436105 10777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:47.436163 10777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:47.445423 10777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:37497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:37497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:47.446632 10777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:47.447799 10777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:47.458582 10782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:47.458629 10788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:47.458658 10783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:47.459551 10777 server_base.cc:1034] running on GCE node
I20260807 03:02:47.460213 10777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:47.461436 10777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:47.462589 10777 hybrid_clock.cc:648] HybridClock initialized: now 1786071767462543 us; error 32 us; skew 500 ppm
I20260807 03:02:47.462898 10777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:47 dist-test-slave-fgb3 krb5kdc[10756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071767, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:47.465823 10777 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:47.467133 10777 webserver.cc:460] Webserver started at http://127.5.122.62:43869/ using document root <none> and password file <none>
I20260807 03:02:47.467701 10777 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:47.467797 10777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:47.468048 10777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:47.470731 10777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "e3dc69cf43e54fe6959cf1d580336f2b"
format_stamp: "Formatted at 2026-08-07 03:02:47 on dist-test-slave-fgb3"
server_key: "2ebed8e59806a723806c78e5438a1cde"
server_key_iv: "4c1781e4396a6fde870e084403191f37"
server_key_version: "encryptionkey@0"
I20260807 03:02:47.471424 10777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "e3dc69cf43e54fe6959cf1d580336f2b"
format_stamp: "Formatted at 2026-08-07 03:02:47 on dist-test-slave-fgb3"
server_key: "2ebed8e59806a723806c78e5438a1cde"
server_key_iv: "4c1781e4396a6fde870e084403191f37"
server_key_version: "encryptionkey@0"
I20260807 03:02:47.476064 10777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:02:47.479238 10796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:47.480448 10777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:02:47.480595 10777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "e3dc69cf43e54fe6959cf1d580336f2b"
format_stamp: "Formatted at 2026-08-07 03:02:47 on dist-test-slave-fgb3"
server_key: "2ebed8e59806a723806c78e5438a1cde"
server_key_iv: "4c1781e4396a6fde870e084403191f37"
server_key_version: "encryptionkey@0"
I20260807 03:02:47.480757 10777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:47.510278 10777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:47.514796 10777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:47.515110 10777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:47.530926 10777 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:37497
I20260807 03:02:47.530956 10874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:37497 every 8 connection(s)
I20260807 03:02:47.532182 10777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:47.535971 10875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:47.537020 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 10777
I20260807 03:02:47.537281 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:47.538211 5608 external_mini_cluster.cc:1428] Setting key 0494f2cfb22c8d09aa4652cf69a036f4
I20260807 03:02:47.550856 10875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b: Bootstrap starting.
Aug 07 03:02:47 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:47.554942 10875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:47.556259 10875 log.cc:824] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:47.559852 10875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b: No bootstrap required, opened a new log
I20260807 03:02:47.562394 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:47.540374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.5.122.62:37497)
0807 03:02:47.541152 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:47.541162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:47.541202 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:02:47.542851 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:47.542873 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:47.543252 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:47.543563 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:47.545536 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:47.546871 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:47.548672 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:47.549328 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:47.549450 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:47.554557 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:47.554566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:47.554580 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:47.554618 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:47.558021 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:47.558956 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:47.558963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:47.558969 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:47.559146 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:47.559607 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:47.559610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:47.559612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:47.560202 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:47.560442 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:47.560907 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 03:02:47.561242 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":349,"threads_started":1}
I20260807 03:02:47.568123 10875 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37497 } }
I20260807 03:02:47.568502 10875 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:47.568599 10875 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3dc69cf43e54fe6959cf1d580336f2b, State: Initialized, Role: FOLLOWER
I20260807 03:02:47.569149 10875 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37497 } }
I20260807 03:02:47.569403 10875 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:47.569540 10875 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:47.569700 10875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:47.572299 10875 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37497 } }
I20260807 03:02:47.572849 10875 leader_election.cc:302] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3dc69cf43e54fe6959cf1d580336f2b; no voters:
I20260807 03:02:47.573452 10875 leader_election.cc:288] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:47.573560 10884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:47.574245 10884 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [term 1 LEADER]: Becoming Leader. State: Replica: e3dc69cf43e54fe6959cf1d580336f2b, State: Running, Role: LEADER
I20260807 03:02:47.574854 10884 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37497 } }
I20260807 03:02:47.575721 10875 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:47.577433 10887 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37497 } } }
I20260807 03:02:47.577761 10887 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:47.578663 10892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:47.578476 10888 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3dc69cf43e54fe6959cf1d580336f2b. Latest consensus state: current_term: 1 leader_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3dc69cf43e54fe6959cf1d580336f2b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37497 } } }
I20260807 03:02:47.578824 10888 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:47.584471 10892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:47.593292 10892 catalog_manager.cc:1347] Generated new cluster ID: b0f3b7a6bd3b47038f4c86d82c375da1
I20260807 03:02:47.593428 10892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:47.611627 10892 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:47.612902 10892 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:47.620929 10892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3dc69cf43e54fe6959cf1d580336f2b: Generated new TSK 0
I20260807 03:02:47.621753 10892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:02:47.683894 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37497
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:48.033191 10914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:48.033505 10914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:48.033666 10914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:48.040993 10914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:48.041131 10914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:48.041311 10914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:48.051138 10914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:48.052433 10914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:48.053772 10914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:48.067054 10922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:48.067075 10920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:48.067682 10919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:48.068068 10914 server_base.cc:1034] running on GCE node
I20260807 03:02:48.068686 10914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:48.069730 10914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:48.070938 10914 hybrid_clock.cc:648] HybridClock initialized: now 1786071768070890 us; error 34 us; skew 500 ppm
I20260807 03:02:48.071264 10914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:48 dist-test-slave-fgb3 krb5kdc[10756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071768, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:48.074802 10914 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:48.076683 10914 webserver.cc:460] Webserver started at http://127.5.122.1:39645/ using document root <none> and password file <none>
I20260807 03:02:48.077384 10914 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:48.077497 10914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:48.077803 10914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:48.080812 10914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "45979d17f4ac4e759887ecace30ecf34"
format_stamp: "Formatted at 2026-08-07 03:02:48 on dist-test-slave-fgb3"
server_key: "dab5f6bb6afffab095163221c07b1a90"
server_key_iv: "fd25a966a234684442ccda831acac714"
server_key_version: "encryptionkey@0"
I20260807 03:02:48.081674 10914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "45979d17f4ac4e759887ecace30ecf34"
format_stamp: "Formatted at 2026-08-07 03:02:48 on dist-test-slave-fgb3"
server_key: "dab5f6bb6afffab095163221c07b1a90"
server_key_iv: "fd25a966a234684442ccda831acac714"
server_key_version: "encryptionkey@0"
I20260807 03:02:48.087179 10914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 03:02:48.090586 10931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:48.091852 10914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:02:48.092008 10914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "45979d17f4ac4e759887ecace30ecf34"
format_stamp: "Formatted at 2026-08-07 03:02:48 on dist-test-slave-fgb3"
server_key: "dab5f6bb6afffab095163221c07b1a90"
server_key_iv: "fd25a966a234684442ccda831acac714"
server_key_version: "encryptionkey@0"
I20260807 03:02:48.092211 10914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:48.104061 10914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:48.107795 10914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:48.108049 10914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:48.108968 10914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:48.110256 10914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:48.110369 10914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:48.110488 10914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:48.110543 10914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:48.141458 10914 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:34445
I20260807 03:02:48.141492 11093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:34445 every 8 connection(s)
I20260807 03:02:48.142724 10914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:48.146703 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 10914
I20260807 03:02:48.146961 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:48.147632 5608 external_mini_cluster.cc:1428] Setting key f09fdc9140d5d09abf3c180bea5130ba
Aug 07 03:02:48 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071768, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:48.163115 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:48.145265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:36857 (local address 127.5.122.62:37497)
0807 03:02:48.145702 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:48.145707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:48.146671 (+ 964us) server_negotiation.cc:407] Connection header received
0807 03:02:48.148514 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:48.148520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:48.148602 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:48.148695 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:48.150135 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.150987 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.152789 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.153385 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.153530 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:48.157091 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:48.157096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:48.157099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:48.157130 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:48.160093 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:48.160796 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:48.160802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:48.160805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:48.160972 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:48.161383 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:48.161387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:48.161390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:48.161702 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:48.161875 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:48.162566 (+ 691us) server_negotiation.cc:299] Negotiation successful
0807 03:02:48.162778 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260807 03:02:48.163875 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:48.145784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37497 (local address 127.5.122.1:36857)
0807 03:02:48.146522 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:48.146564 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:48.148264 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:48.148941 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:48.148948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:48.149445 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:48.149980 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:48.149997 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.151192 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.151197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:48.152633 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:48.152646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.153582 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.153586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:48.153770 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:48.154405 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:48.154428 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:48.156882 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:48.160225 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:48.160232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:48.160255 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:48.160675 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:48.161074 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:48.161077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:48.161079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:48.161279 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:48.161836 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:48.161842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:48.162128 (+ 286us) client_negotiation.cc:769] Sending connection context
0807 03:02:48.162414 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 03:02:48.162787 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":292,"threads_started":1}
I20260807 03:02:48.165330 11094 heartbeater.cc:347] Connected to a master server at 127.5.122.62:37497
I20260807 03:02:48.165654 11094 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:48.166543 11094 heartbeater.cc:511] Master 127.5.122.62:37497 requested a full tablet report, sending...
I20260807 03:02:48.168700 10825 ts_manager.cc:194] Registered new tserver with Master: 45979d17f4ac4e759887ecace30ecf34 (127.5.122.1:34445)
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
I20260807 03:02:48.170195 10825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:36857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:02:48.193130 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37497
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:48.507130 11103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:48.507401 11103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:48.507541 11103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:48.514427 11103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:48.514537 11103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:48.514693 11103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:48.524061 11103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:48.525375 11103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:48.526654 11103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:48.539803 11109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:48.539798 11110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:48.540513 11112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:48.541275 11103 server_base.cc:1034] running on GCE node
I20260807 03:02:48.541817 11103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:48.542732 11103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:48.543948 11103 hybrid_clock.cc:648] HybridClock initialized: now 1786071768543899 us; error 40 us; skew 500 ppm
I20260807 03:02:48.544258 11103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:48 dist-test-slave-fgb3 krb5kdc[10756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071768, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:48.547271 11103 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:48.548581 11103 webserver.cc:460] Webserver started at http://127.5.122.2:43909/ using document root <none> and password file <none>
I20260807 03:02:48.549209 11103 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:48.549330 11103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:48.549614 11103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:48.552400 11103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "72ae45843f9540818d2a008c26485eca"
format_stamp: "Formatted at 2026-08-07 03:02:48 on dist-test-slave-fgb3"
server_key: "f8c0ac8dd9ef3e3f816cbfb7bb0a89d3"
server_key_iv: "15ef1d3b2591fd61ac3c0a427bf380c0"
server_key_version: "encryptionkey@0"
I20260807 03:02:48.553129 11103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "72ae45843f9540818d2a008c26485eca"
format_stamp: "Formatted at 2026-08-07 03:02:48 on dist-test-slave-fgb3"
server_key: "f8c0ac8dd9ef3e3f816cbfb7bb0a89d3"
server_key_iv: "15ef1d3b2591fd61ac3c0a427bf380c0"
server_key_version: "encryptionkey@0"
I20260807 03:02:48.557904 11103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 03:02:48.561285 11121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:48.562531 11103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:02:48.562676 11103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "72ae45843f9540818d2a008c26485eca"
format_stamp: "Formatted at 2026-08-07 03:02:48 on dist-test-slave-fgb3"
server_key: "f8c0ac8dd9ef3e3f816cbfb7bb0a89d3"
server_key_iv: "15ef1d3b2591fd61ac3c0a427bf380c0"
server_key_version: "encryptionkey@0"
I20260807 03:02:48.562855 11103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:48.576529 11103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:48.580440 11103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:48.580664 11103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:48.581532 11103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:48.582901 11103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:48.582979 11103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:48.583073 11103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:48.583117 11103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:48.613333 11103 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:37137
I20260807 03:02:48.613390 11279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:37137 every 8 connection(s)
I20260807 03:02:48.614631 11103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:48.620891 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 11103
I20260807 03:02:48.621094 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:48.621629 5608 external_mini_cluster.cc:1428] Setting key d2ea86a7f3c51415ab46959d9120a3f9
Aug 07 03:02:48 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071768, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:48.635419 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:48.617134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:57459 (local address 127.5.122.62:37497)
0807 03:02:48.617307 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:48.617312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:48.618400 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 03:02:48.620128 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:48.620133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:48.620201 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:48.620285 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:48.622181 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.623026 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.625095 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.625560 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.625663 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:48.629582 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:48.629587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:48.629590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:48.629622 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:48.632314 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:48.632999 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:48.633003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:48.633005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:48.633130 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:48.633643 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:48.633647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:48.633649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:48.633931 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:48.634085 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:48.634810 (+ 725us) server_negotiation.cc:299] Negotiation successful
0807 03:02:48.635035 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 03:02:48.636368 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:48.617518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37497 (local address 127.5.122.2:57459)
0807 03:02:48.618225 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:48.618262 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:48.619897 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:48.620466 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:48.620473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:48.621255 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:48.622006 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:48.622031 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.623184 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.623189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:48.624928 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:48.624943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:48.625801 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:48.625809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:48.626075 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:48.626749 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:48.626790 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:48.629359 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:48.632443 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:48.632450 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:48.632463 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:48.632890 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:48.633264 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:48.633268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:48.633271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:48.633503 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:48.634066 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:48.634074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:48.634444 (+ 370us) client_negotiation.cc:769] Sending connection context
0807 03:02:48.634699 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 03:02:48.635225 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":271,"threads_started":1}
I20260807 03:02:48.637895 11280 heartbeater.cc:347] Connected to a master server at 127.5.122.62:37497
I20260807 03:02:48.638233 11280 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:48.638978 11280 heartbeater.cc:511] Master 127.5.122.62:37497 requested a full tablet report, sending...
I20260807 03:02:48.640475 10825 ts_manager.cc:194] Registered new tserver with Master: 72ae45843f9540818d2a008c26485eca (127.5.122.2:37137)
I20260807 03:02:48.641575 10825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:57459
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:48.667521 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37497
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:48.980444 11290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:48.980733 11290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:48.980885 11290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:48.988153 11290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:48.988276 11290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:48.988430 11290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:48.998160 11290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:48.999452 11290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:49.000756 11290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:49.013038 11298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:49.013079 11301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:49.013018 11299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:49.014114 11290 server_base.cc:1034] running on GCE node
I20260807 03:02:49.014742 11290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:49.015645 11290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:49.016839 11290 hybrid_clock.cc:648] HybridClock initialized: now 1786071769016792 us; error 42 us; skew 500 ppm
I20260807 03:02:49.017138 11290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[10756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:49.020388 11290 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:49.021791 11290 webserver.cc:460] Webserver started at http://127.5.122.3:45885/ using document root <none> and password file <none>
I20260807 03:02:49.022512 11290 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:49.022635 11290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:49.022946 11290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:49.025874 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "b6313ac357cb480db62e5b117ac16898"
format_stamp: "Formatted at 2026-08-07 03:02:49 on dist-test-slave-fgb3"
server_key: "68c3d7e5e81e59f9bc60847f1b4a5f6e"
server_key_iv: "29b20e349bb7fd6404f17582bd8779b0"
server_key_version: "encryptionkey@0"
I20260807 03:02:49.026703 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "b6313ac357cb480db62e5b117ac16898"
format_stamp: "Formatted at 2026-08-07 03:02:49 on dist-test-slave-fgb3"
server_key: "68c3d7e5e81e59f9bc60847f1b4a5f6e"
server_key_iv: "29b20e349bb7fd6404f17582bd8779b0"
server_key_version: "encryptionkey@0"
I20260807 03:02:49.031664 11290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:02:49.035108 11313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:49.036343 11290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:49.036494 11290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "b6313ac357cb480db62e5b117ac16898"
format_stamp: "Formatted at 2026-08-07 03:02:49 on dist-test-slave-fgb3"
server_key: "68c3d7e5e81e59f9bc60847f1b4a5f6e"
server_key_iv: "29b20e349bb7fd6404f17582bd8779b0"
server_key_version: "encryptionkey@0"
I20260807 03:02:49.036672 11290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:49.052913 11290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:49.056797 11290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:49.057060 11290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:49.058043 11290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:49.059420 11290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:49.059499 11290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:49.059625 11290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:49.059679 11290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:49.091534 11290 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:33175
I20260807 03:02:49.091557 11469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:33175 every 8 connection(s)
I20260807 03:02:49.092864 11290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:49.095285 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 11290
I20260807 03:02:49.095507 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:49.096077 5608 external_mini_cluster.cc:1428] Setting key 42e9fdcfc23473d3964aae5531607544
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:49.111835 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:49.095358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53793 (local address 127.5.122.62:37497)
0807 03:02:49.095575 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:49.095582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:49.097024 (+ 1442us) server_negotiation.cc:407] Connection header received
0807 03:02:49.099281 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:49.099287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:49.099377 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:49.099458 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:49.100810 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.101534 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.103035 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.103429 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.103516 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:49.106584 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:49.106589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:49.106592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:49.106624 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:49.108943 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.109676 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.109680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.109682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.109814 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.110192 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.110196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.110198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.110547 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:49.110674 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:49.111414 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 03:02:49.111574 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 03:02:49.112694 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:49.095981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37497 (local address 127.5.122.3:53793)
0807 03:02:49.096834 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:49.096895 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:49.099016 (+ 2121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:49.099640 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:49.099647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:49.100154 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:49.100680 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:49.100696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.101651 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.101655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:49.102929 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:49.102941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.103593 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.103598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:49.103793 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:49.104345 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:49.104365 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:49.106400 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:49.109054 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:49.109060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:49.109070 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:49.109560 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:49.109915 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:49.109920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:49.109924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:49.110099 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:49.110676 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:49.110683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:49.111024 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 03:02:49.111326 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 03:02:49.111690 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":322,"threads_started":1}
I20260807 03:02:49.113842 11470 heartbeater.cc:347] Connected to a master server at 127.5.122.62:37497
I20260807 03:02:49.114111 11470 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:49.114831 11470 heartbeater.cc:511] Master 127.5.122.62:37497 requested a full tablet report, sending...
I20260807 03:02:49.116255 10825 ts_manager.cc:194] Registered new tserver with Master: b6313ac357cb480db62e5b117ac16898 (127.5.122.3:33175)
I20260807 03:02:49.117071 10825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:53793
I20260807 03:02:49.123261 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[10756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:49.172864 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:49.158662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.5.122.1:34445)
0807 03:02:49.159291 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:49.159298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:49.159326 (+ 28us) server_negotiation.cc:407] Connection header received
0807 03:02:49.159386 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:49.159393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:49.159595 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:49.159716 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:49.160980 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.161978 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.163241 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.163807 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.163918 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:49.167678 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:49.167689 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:49.167704 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:49.167738 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:49.170497 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.171059 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.171066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.171068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.171185 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.171466 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.171469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.171471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.171790 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:49.172020 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:49.172294 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 03:02:49.172487 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":315,"threads_started":1}
I20260807 03:02:49.173177 11094 heartbeater.cc:503] Master 127.5.122.62:37497 was elected leader, sending a full tablet report...
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:49.189733 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:49.177230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.5.122.62:37497)
0807 03:02:49.177434 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:49.177440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:49.177617 (+ 177us) server_negotiation.cc:407] Connection header received
0807 03:02:49.177737 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:49.177741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:49.177811 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:49.177889 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:49.179191 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.179921 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.181094 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.181589 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.181681 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:49.185272 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:49.185277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:49.185280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:49.185314 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:49.187585 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.188165 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.188169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.188171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.188282 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.188556 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.188559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.188561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.188881 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:49.189056 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:49.189308 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 03:02:49.189459 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 03:02:49.190349 10825 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55014
I20260807 03:02:49.191295 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10914
I20260807 03:02:49.202962 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:49.192162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.5.122.1:34445)
0807 03:02:49.192371 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:49.192377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:49.192455 (+ 78us) server_negotiation.cc:407] Connection header received
0807 03:02:49.192573 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:49.192577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:49.192637 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:49.192712 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:49.193817 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.194535 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.195738 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.196168 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.196289 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:49.198433 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:49.198438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:49.198441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:49.198472 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:49.200752 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.201349 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.201355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.201359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.201493 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.201791 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.201794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.201795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.202038 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:49.202220 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:49.202463 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 03:02:49.202632 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 03:02:49.203634 11088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:49.284490 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 10914
I20260807 03:02:49.302294 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 11103
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:02:49.319588 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:49.303352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60334 (local address 127.5.122.2:37137)
0807 03:02:49.303929 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:49.303936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:49.303965 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:02:49.304029 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:49.304035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:49.304237 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:49.304380 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:49.305455 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.306562 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.307793 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.308384 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.308495 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:49.312924 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:49.312942 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:49.312960 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:49.313004 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:49.316905 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.317647 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.317655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.317657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.317791 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.318107 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.318110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.318112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.318619 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:49.318867 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:49.319087 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 03:02:49.319291 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":300,"threads_started":1}
I20260807 03:02:49.320874 11275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:49.402249 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 11103
I20260807 03:02:49.423700 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 11290
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[10756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:02:49.438807 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:49.424719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.5.122.3:33175)
0807 03:02:49.425363 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:49.425372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:49.425401 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:02:49.425465 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:49.425471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:49.425677 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:49.425801 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:49.427071 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.428086 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.429233 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:49.429755 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:49.429847 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:49.433850 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:49.433871 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:49.433886 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:49.433919 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:49.436448 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.437054 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.437060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.437063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.437174 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:49.437454 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:49.437456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:49.437458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:49.437769 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:49.438011 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:49.438213 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 03:02:49.438449 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":331,"threads_started":1}
I20260807 03:02:49.440088 11464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:49.523730 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 11290
I20260807 03:02:49.541445 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 10777
I20260807 03:02:49.542707 10869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:49.643633 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 10777
2026-08-07T03:02:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3916 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[11500](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[11500](info): set up 2 sockets
Aug 07 03:02:49 dist-test-slave-fgb3 krb5kdc[11500](info): commencing operation
krb5kdc: starting...
W20260807 03:02:50.824254 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:02:50 dist-test-slave-fgb3 krb5kdc[11500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:02:51.019609 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44635
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:44635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:51.335114 11516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:51.335402 11516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:51.335541 11516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:51.342732 11516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:51.342856 11516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:51.342933 11516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:51.343011 11516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:51.343076 11516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:51.352739 11516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:44635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:51.354003 11516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:51.355258 11516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:51.365813 11524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:51.366019 11525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:51.366153 11529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:51.366842 11516 server_base.cc:1034] running on GCE node
I20260807 03:02:51.367522 11516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:51.368805 11516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:51.370023 11516 hybrid_clock.cc:648] HybridClock initialized: now 1786071771369979 us; error 33 us; skew 500 ppm
I20260807 03:02:51.370419 11516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:51 dist-test-slave-fgb3 krb5kdc[11500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071771, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:51.373332 11516 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:02:51.374408 11516 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:51.374511 11516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:51.374809 11516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:51.377576 11516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "72a4730013fe4c3a87d0a32a42933c3b"
format_stamp: "Formatted at 2026-08-07 03:02:51 on dist-test-slave-fgb3"
server_key: "77d66988e6cb2cd52a4e758abdbf2b16"
server_key_iv: "289afc6ddbd195f711dbf6c0e034922b"
server_key_version: "encryptionkey@0"
I20260807 03:02:51.378454 11516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "72a4730013fe4c3a87d0a32a42933c3b"
format_stamp: "Formatted at 2026-08-07 03:02:51 on dist-test-slave-fgb3"
server_key: "77d66988e6cb2cd52a4e758abdbf2b16"
server_key_iv: "289afc6ddbd195f711dbf6c0e034922b"
server_key_version: "encryptionkey@0"
I20260807 03:02:51.383276 11516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:02:51.386451 11537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:51.387573 11516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:02:51.387717 11516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "72a4730013fe4c3a87d0a32a42933c3b"
format_stamp: "Formatted at 2026-08-07 03:02:51 on dist-test-slave-fgb3"
server_key: "77d66988e6cb2cd52a4e758abdbf2b16"
server_key_iv: "289afc6ddbd195f711dbf6c0e034922b"
server_key_version: "encryptionkey@0"
I20260807 03:02:51.387907 11516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:51.424715 11516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:51.428505 11516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:51.428730 11516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:51.445585 11516 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:44635
I20260807 03:02:51.445593 11606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:44635 every 8 connection(s)
I20260807 03:02:51.446460 11516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:51.446974 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 11516
I20260807 03:02:51.447192 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:51.447925 5608 external_mini_cluster.cc:1428] Setting key 5dfc43a2cce106ff00645fa09795013c
I20260807 03:02:51.450575 11607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 03:02:51 dist-test-slave-fgb3 krb5kdc[11500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:51.462982 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b: Bootstrap starting.
I20260807 03:02:51.466698 11607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:51.467900 11607 log.cc:824] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:51.471055 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b: No bootstrap required, opened a new log
I20260807 03:02:51.471556 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:51.449770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.5.122.62:44635)
0807 03:02:51.450632 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:51.450648 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:51.450695 (+ 47us) server_negotiation.cc:407] Connection header received
0807 03:02:51.452899 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:51.452927 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:51.453419 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:51.453839 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:51.455173 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:51.457140 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:51.458462 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:51.459424 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:51.459623 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:51.463801 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:51.463811 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:51.463827 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:51.463875 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:51.467458 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:51.468086 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:51.468093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:51.468100 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:51.468295 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:51.468698 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:51.468702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:51.468704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:51.469280 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:51.469555 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:51.469900 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 03:02:51.470295 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":337,"threads_started":1}
I20260807 03:02:51.478744 11607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a4730013fe4c3a87d0a32a42933c3b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44635 } }
I20260807 03:02:51.479049 11607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:51.479120 11607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a4730013fe4c3a87d0a32a42933c3b, State: Initialized, Role: FOLLOWER
I20260807 03:02:51.479558 11607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a4730013fe4c3a87d0a32a42933c3b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44635 } }
I20260807 03:02:51.479808 11607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:51.479940 11607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:51.480119 11607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:51.482372 11607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a4730013fe4c3a87d0a32a42933c3b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44635 } }
I20260807 03:02:51.482810 11607 leader_election.cc:302] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a4730013fe4c3a87d0a32a42933c3b; no voters:
I20260807 03:02:51.483381 11607 leader_election.cc:288] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:51.483515 11617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:51.484339 11617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [term 1 LEADER]: Becoming Leader. State: Replica: 72a4730013fe4c3a87d0a32a42933c3b, State: Running, Role: LEADER
I20260807 03:02:51.484828 11617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a4730013fe4c3a87d0a32a42933c3b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44635 } }
I20260807 03:02:51.485325 11607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:51.487551 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a4730013fe4c3a87d0a32a42933c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a4730013fe4c3a87d0a32a42933c3b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44635 } } }
I20260807 03:02:51.488073 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:51.488210 11619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a4730013fe4c3a87d0a32a42933c3b. Latest consensus state: current_term: 1 leader_uuid: "72a4730013fe4c3a87d0a32a42933c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a4730013fe4c3a87d0a32a42933c3b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44635 } } }
I20260807 03:02:51.488539 11619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:51.490154 11628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:51.495129 11628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:51.503885 11628 catalog_manager.cc:1347] Generated new cluster ID: b66e0cf4ddd8427aaeea050d641660d3
I20260807 03:02:51.503996 11628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:51.527446 11628 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:51.528702 11628 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:51.536796 11628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b: Generated new TSK 0
I20260807 03:02:51.537451 11628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:02:51.587435 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44635
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:51.900642 11649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:51.900924 11649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:51.901073 11649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:51.908294 11649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:51.908406 11649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:51.908569 11649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:51.918205 11649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:51.919467 11649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:51.920758 11649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:51.933671 11656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:51.933712 11658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:51.933624 11655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:51.935313 11649 server_base.cc:1034] running on GCE node
I20260807 03:02:51.935849 11649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:51.936704 11649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:51.937893 11649 hybrid_clock.cc:648] HybridClock initialized: now 1786071771937839 us; error 49 us; skew 500 ppm
I20260807 03:02:51.938199 11649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:51 dist-test-slave-fgb3 krb5kdc[11500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071771, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:51.941571 11649 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:02:51.942690 11649 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:51.942788 11649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:51.943033 11649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:51.945763 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "4bfbb60590ed4b67bfde22ab0445a176"
format_stamp: "Formatted at 2026-08-07 03:02:51 on dist-test-slave-fgb3"
server_key: "764c01e35e226165d31711c369ffc592"
server_key_iv: "e80745caf30af05814b6ed6bcffe9e32"
server_key_version: "encryptionkey@0"
I20260807 03:02:51.946607 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "4bfbb60590ed4b67bfde22ab0445a176"
format_stamp: "Formatted at 2026-08-07 03:02:51 on dist-test-slave-fgb3"
server_key: "764c01e35e226165d31711c369ffc592"
server_key_iv: "e80745caf30af05814b6ed6bcffe9e32"
server_key_version: "encryptionkey@0"
I20260807 03:02:51.951372 11649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:02:51.954766 11666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:51.956024 11649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 03:02:51.956171 11649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "4bfbb60590ed4b67bfde22ab0445a176"
format_stamp: "Formatted at 2026-08-07 03:02:51 on dist-test-slave-fgb3"
server_key: "764c01e35e226165d31711c369ffc592"
server_key_iv: "e80745caf30af05814b6ed6bcffe9e32"
server_key_version: "encryptionkey@0"
I20260807 03:02:51.956398 11649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:51.978559 11649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:51.982635 11649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:51.982863 11649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:51.983803 11649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:51.985087 11649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:51.985178 11649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:51.985255 11649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:51.985312 11649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:52.017839 11649 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:33755
I20260807 03:02:52.017943 11839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:33755 every 8 connection(s)
I20260807 03:02:52.018741 11649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:52.027743 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 11649
I20260807 03:02:52.027948 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:52.028563 5608 external_mini_cluster.cc:1428] Setting key 5c662bc974084b4ff93d3be943d5efb8
Aug 07 03:02:52 dist-test-slave-fgb3 krb5kdc[11500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071771, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:52.037828 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:52.021132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:36131 (local address 127.5.122.62:44635)
0807 03:02:52.021578 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:52.021582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:52.022512 (+ 930us) server_negotiation.cc:407] Connection header received
0807 03:02:52.024344 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:52.024350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:52.024428 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:52.024524 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:52.026069 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.026843 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.028437 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.028858 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.028969 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:52.032345 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:52.032350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:52.032353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:52.032384 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:52.034764 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:52.035509 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:52.035514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:52.035516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:52.035632 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:52.036081 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:52.036086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:52.036090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:52.036445 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:52.036610 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:52.037284 (+ 674us) server_negotiation.cc:299] Negotiation successful
0807 03:02:52.037491 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":240,"threads_started":1}
I20260807 03:02:52.038779 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:52.021590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44635 (local address 127.5.122.1:36131)
0807 03:02:52.022356 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:52.022395 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:52.024065 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:52.024756 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:52.024772 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:52.025325 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:52.025879 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:52.025897 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.026989 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.026993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:52.028299 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:52.028312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.029075 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.029083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:52.029294 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:52.029915 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:52.029943 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:52.032138 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:52.034888 (+ 2750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:52.034897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:52.034912 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:52.035404 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:52.035733 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:52.035739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:52.035742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:52.035949 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:52.036589 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:52.036597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:52.036930 (+ 333us) client_negotiation.cc:769] Sending connection context
0807 03:02:52.037159 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 03:02:52.037542 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":294,"threads_started":1}
I20260807 03:02:52.040290 11840 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44635
I20260807 03:02:52.040573 11840 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:52.041186 11840 heartbeater.cc:511] Master 127.5.122.62:44635 requested a full tablet report, sending...
I20260807 03:02:52.043113 11561 ts_manager.cc:194] Registered new tserver with Master: 4bfbb60590ed4b67bfde22ab0445a176 (127.5.122.1:33755)
I20260807 03:02:52.044544 11561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:36131
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:02:52.074420 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44635
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:52.389894 11848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:52.390170 11848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:52.390336 11848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:52.397466 11848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:52.397588 11848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:52.397746 11848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:52.407611 11848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:52.408955 11848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:52.410249 11848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:52.422722 11854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:52.422777 11856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:52.422830 11853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:52.423338 11848 server_base.cc:1034] running on GCE node
I20260807 03:02:52.423936 11848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:52.424821 11848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:52.426031 11848 hybrid_clock.cc:648] HybridClock initialized: now 1786071772425974 us; error 37 us; skew 500 ppm
I20260807 03:02:52.426379 11848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:52 dist-test-slave-fgb3 krb5kdc[11500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071772, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:52.429284 11848 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:02:52.430464 11848 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:52.430572 11848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:52.430886 11848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:52.433880 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "6235eb5c57124efbb9c2eeaac23a2293"
format_stamp: "Formatted at 2026-08-07 03:02:52 on dist-test-slave-fgb3"
server_key: "12678989c126709834bcb3e7f457bd41"
server_key_iv: "9a65e165317a0190392a55f3dff8a074"
server_key_version: "encryptionkey@0"
I20260807 03:02:52.434723 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "6235eb5c57124efbb9c2eeaac23a2293"
format_stamp: "Formatted at 2026-08-07 03:02:52 on dist-test-slave-fgb3"
server_key: "12678989c126709834bcb3e7f457bd41"
server_key_iv: "9a65e165317a0190392a55f3dff8a074"
server_key_version: "encryptionkey@0"
I20260807 03:02:52.439651 11848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:02:52.442937 11864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:52.444095 11848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260807 03:02:52.444245 11848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "6235eb5c57124efbb9c2eeaac23a2293"
format_stamp: "Formatted at 2026-08-07 03:02:52 on dist-test-slave-fgb3"
server_key: "12678989c126709834bcb3e7f457bd41"
server_key_iv: "9a65e165317a0190392a55f3dff8a074"
server_key_version: "encryptionkey@0"
I20260807 03:02:52.444455 11848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:52.470084 11848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:52.473654 11848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:52.473893 11848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:52.474800 11848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:52.476011 11848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:52.476107 11848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:52.476210 11848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:52.476281 11848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:52.506345 11848 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:39909
I20260807 03:02:52.506402 12015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:39909 every 8 connection(s)
I20260807 03:02:52.507125 11848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:52.513598 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 11848
I20260807 03:02:52.513780 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:52.514416 5608 external_mini_cluster.cc:1428] Setting key 384da3a3eb0c5ab21e9699cdde7d976b
Aug 07 03:02:52 dist-test-slave-fgb3 krb5kdc[11500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071772, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:52.525986 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:52.509234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:50405 (local address 127.5.122.62:44635)
0807 03:02:52.509439 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:52.509445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:52.510559 (+ 1114us) server_negotiation.cc:407] Connection header received
0807 03:02:52.512251 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:52.512255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:52.512322 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:52.512407 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:52.513783 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.514884 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.516570 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.517165 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.517284 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:52.520486 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:52.520491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:52.520495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:52.520529 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:52.523263 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:52.523880 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:52.523884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:52.523888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:52.524038 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:52.524426 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:52.524430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:52.524432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:52.524679 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:52.524818 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:52.525423 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 03:02:52.525637 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:02:52.526904 12019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:52.509716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44635 (local address 127.5.122.2:50405)
0807 03:02:52.510422 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:52.510460 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:52.512024 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:52.512586 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:52.512594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:52.513060 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:52.513597 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:52.513612 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.515041 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.515046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:52.516438 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:52.516450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:52.517347 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:52.517352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:52.517537 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:52.518150 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:52.518172 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:52.520284 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:52.523378 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:52.523384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:52.523394 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:52.523773 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:52.524131 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:52.524134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:52.524136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:52.524314 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:52.524803 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:52.524811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:52.525095 (+ 284us) client_negotiation.cc:769] Sending connection context
0807 03:02:52.525348 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 03:02:52.525801 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":276,"threads_started":1}
I20260807 03:02:52.528187 12016 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44635
I20260807 03:02:52.528532 12016 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:52.529218 12016 heartbeater.cc:511] Master 127.5.122.62:44635 requested a full tablet report, sending...
I20260807 03:02:52.530774 11561 ts_manager.cc:194] Registered new tserver with Master: 6235eb5c57124efbb9c2eeaac23a2293 (127.5.122.2:39909)
I20260807 03:02:52.531821 11561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:50405
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:02:52.561333 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44635
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:02:52.879928 12022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:52.880180 12022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:52.880275 12022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:52.887235 12022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:52.887358 12022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:52.887530 12022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:52.897416 12022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:52.898768 12022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:52.900105 12022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:52.912127 12032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:52.912207 12029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:52.912180 12030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:52.913017 12022 server_base.cc:1034] running on GCE node
I20260807 03:02:52.913595 12022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:52.914487 12022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:52.915679 12022 hybrid_clock.cc:648] HybridClock initialized: now 1786071772915626 us; error 47 us; skew 500 ppm
I20260807 03:02:52.915969 12022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:02:52 dist-test-slave-fgb3 krb5kdc[11500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071772, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:02:52.919016 12022 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:02:52.920431 12022 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:52.920532 12022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:52.920835 12022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:52.923648 12022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "fba352f9002541c8b0bc35d2e9523a22"
format_stamp: "Formatted at 2026-08-07 03:02:52 on dist-test-slave-fgb3"
server_key: "7d7ad24762dff26cdbf9dec3105bf342"
server_key_iv: "f378dd0d4cc8f9ef6cb2ddc419045a35"
server_key_version: "encryptionkey@0"
I20260807 03:02:52.924432 12022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "fba352f9002541c8b0bc35d2e9523a22"
format_stamp: "Formatted at 2026-08-07 03:02:52 on dist-test-slave-fgb3"
server_key: "7d7ad24762dff26cdbf9dec3105bf342"
server_key_iv: "f378dd0d4cc8f9ef6cb2ddc419045a35"
server_key_version: "encryptionkey@0"
I20260807 03:02:52.929117 12022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 03:02:52.933068 12040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:52.934976 12022 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 03:02:52.935140 12022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "fba352f9002541c8b0bc35d2e9523a22"
format_stamp: "Formatted at 2026-08-07 03:02:52 on dist-test-slave-fgb3"
server_key: "7d7ad24762dff26cdbf9dec3105bf342"
server_key_iv: "f378dd0d4cc8f9ef6cb2ddc419045a35"
server_key_version: "encryptionkey@0"
I20260807 03:02:52.935377 12022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:52.963706 12022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:52.967698 12022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:52.967943 12022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:52.968844 12022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:52.970134 12022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:52.970223 12022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:52.970373 12022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:52.970453 12022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:53.000001 12022 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36605
I20260807 03:02:53.000073 12198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36605 every 8 connection(s)
I20260807 03:02:53.000840 12022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:53.001097 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 12022
I20260807 03:02:53.001281 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:53.001863 5608 external_mini_cluster.cc:1428] Setting key 5750f86d48f5d846f1d3f4e93a71d968
Aug 07 03:02:53 dist-test-slave-fgb3 krb5kdc[11500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071772, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:02:53.020123 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.003385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:49337 (local address 127.5.122.62:44635)
0807 03:02:53.003579 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.003584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.004957 (+ 1373us) server_negotiation.cc:407] Connection header received
0807 03:02:53.006821 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.006825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.006892 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.006970 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:53.008331 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.009033 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.010561 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.010972 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.011062 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.014167 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:53.014172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:53.014175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:53.014205 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:53.017034 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.017818 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.017823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.017826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.018002 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.018447 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.018452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.018456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.018795 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:53.018939 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.019581 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.019792 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:02:53.020851 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.003957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44635 (local address 127.5.122.3:49337)
0807 03:02:53.004797 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:53.004850 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:53.006597 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:53.007151 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:53.007158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:53.007661 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:53.008207 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.008223 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.009154 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.009157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.010431 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.010444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.011133 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.011137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.011340 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.011893 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:02:53.011914 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:53.013997 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:53.017226 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:53.017235 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:53.017274 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:53.017707 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:53.018127 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:02:53.018132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:02:53.018135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:02:53.018351 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:02:53.018902 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:53.018907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:53.019197 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 03:02:53.019429 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 03:02:53.019790 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":368,"threads_started":1}
I20260807 03:02:53.022163 12201 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44635
I20260807 03:02:53.022478 12201 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:53.023065 12201 heartbeater.cc:511] Master 127.5.122.62:44635 requested a full tablet report, sending...
I20260807 03:02:53.024371 11561 ts_manager.cc:194] Registered new tserver with Master: fba352f9002541c8b0bc35d2e9523a22 (127.5.122.3:36605)
I20260807 03:02:53.025122 11561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:49337
I20260807 03:02:53.029191 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:02:53.044852 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.033496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.5.122.62:44635)
0807 03:02:53.033671 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.033675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.034167 (+ 492us) server_negotiation.cc:407] Connection header received
0807 03:02:53.034286 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.034289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.034372 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.034446 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:53.035766 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.036430 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.037606 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.038044 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.038132 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.040192 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:53.040198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:53.040202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:53.040231 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:53.042734 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.043382 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.043397 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.043399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.043513 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.043773 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.043776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.043778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.044058 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:53.044222 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.044479 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.044621 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:02:53.047315 11840 heartbeater.cc:503] Master 127.5.122.62:44635 was elected leader, sending a full tablet report...
I20260807 03:02:53.048653 11561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:53.055260 11561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:53.077536 12227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.067517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.5.122.2:39909)
0807 03:02:53.068088 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.068098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.069576 (+ 1478us) server_negotiation.cc:407] Connection header received
0807 03:02:53.069861 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.069868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.070083 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.070236 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:53.071026 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.072431 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.075009 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.076385 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.076515 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.076710 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.077063 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.077230 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":279,"threads_started":1}
I20260807 03:02:53.077809 12225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.066631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:33755 (local address 127.0.0.1:36488)
0807 03:02:53.068690 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:53.068736 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:53.068908 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:53.069965 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:53.069972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:53.070001 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:53.070440 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.070451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.072362 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.072367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.074775 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.074798 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.076878 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.076883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.077042 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.077074 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:02:53.077161 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 03:02:53.077361 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1802,"mutex_wait_us":3,"thread_start_us":2564,"threads_started":1}
I20260807 03:02:53.078523 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.066791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36488 (local address 127.5.122.1:33755)
0807 03:02:53.069318 (+ 2527us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.069326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.069355 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:02:53.069455 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.069462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.069708 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.070643 (+ 935us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:53.070866 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.072207 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.074935 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.076389 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.076515 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.076737 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.078058 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.078222 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2396,"thread_start_us":2251,"threads_started":1}
I20260807 03:02:53.078537 12230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.070188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36605 (local address 127.0.0.1:37384)
0807 03:02:53.071365 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:53.071381 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:53.071495 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:53.072021 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:53.072026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:53.072049 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:53.072482 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.072497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.074407 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.074412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.075889 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.075901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.077701 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.077709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.077881 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.077909 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 03:02:53.077984 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 03:02:53.078117 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":35,"thread_start_us":932,"threads_started":1}
I20260807 03:02:53.078662 12223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.067010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39909 (local address 127.0.0.1:47020)
0807 03:02:53.069471 (+ 2461us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:53.069488 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:53.069676 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:53.070439 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:53.070444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:53.070468 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:53.070844 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.070855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.072574 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.072579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.074870 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.074884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.076573 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.076580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.076758 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.076807 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 03:02:53.078171 (+ 1364us) client_negotiation.cc:240] Negotiation successful
0807 03:02:53.078291 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2334,"mutex_wait_us":318,"thread_start_us":1260,"threads_started":1}
I20260807 03:02:53.079278 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.069838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.5.122.3:36605)
0807 03:02:53.070922 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.070930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.071486 (+ 556us) server_negotiation.cc:407] Connection header received
0807 03:02:53.071630 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.071637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.071838 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.072021 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:53.072676 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.074234 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.076034 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.077206 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.077328 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.077540 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.078805 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.078969 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":922,"thread_start_us":774,"threads_started":1}
I20260807 03:02:53.081753 11926 tablet_service.cc:1467] Processing CreateTablet for tablet dc9944a43b2a4946990955244a401871 (DEFAULT_TABLE table=test-table [id=422c1a3754414797915e5d2eb4cd880b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:53.082050 11747 tablet_service.cc:1467] Processing CreateTablet for tablet dc9944a43b2a4946990955244a401871 (DEFAULT_TABLE table=test-table [id=422c1a3754414797915e5d2eb4cd880b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:53.082525 12106 tablet_service.cc:1467] Processing CreateTablet for tablet dc9944a43b2a4946990955244a401871 (DEFAULT_TABLE table=test-table [id=422c1a3754414797915e5d2eb4cd880b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:53.083235 11926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9944a43b2a4946990955244a401871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:53.083446 11747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9944a43b2a4946990955244a401871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:53.083838 12106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9944a43b2a4946990955244a401871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:53.097635 12233 tablet_bootstrap.cc:492] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Bootstrap starting.
I20260807 03:02:53.099623 12234 tablet_bootstrap.cc:492] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Bootstrap starting.
I20260807 03:02:53.101043 12232 tablet_bootstrap.cc:492] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Bootstrap starting.
I20260807 03:02:53.101755 12233 tablet_bootstrap.cc:654] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:53.102792 12233 log.cc:824] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:53.102994 12234 tablet_bootstrap.cc:654] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:53.104197 12234 log.cc:824] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:53.104333 12232 tablet_bootstrap.cc:654] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:53.105361 12232 log.cc:824] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:53.106099 12233 tablet_bootstrap.cc:492] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: No bootstrap required, opened a new log
I20260807 03:02:53.106413 12233 ts_tablet_manager.cc:1397] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260807 03:02:53.107271 12234 tablet_bootstrap.cc:492] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: No bootstrap required, opened a new log
I20260807 03:02:53.107517 12234 ts_tablet_manager.cc:1397] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 03:02:53.108178 12232 tablet_bootstrap.cc:492] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: No bootstrap required, opened a new log
I20260807 03:02:53.108421 12232 ts_tablet_manager.cc:1397] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 03:02:53.114276 12234 raft_consensus.cc:348] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.114753 12234 raft_consensus.cc:374] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:53.114864 12234 raft_consensus.cc:729] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fba352f9002541c8b0bc35d2e9523a22, State: Initialized, Role: FOLLOWER
I20260807 03:02:53.115551 12234 consensus_queue.cc:260] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.116585 12201 heartbeater.cc:503] Master 127.5.122.62:44635 was elected leader, sending a full tablet report...
I20260807 03:02:53.116603 12233 raft_consensus.cc:348] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.117002 12233 raft_consensus.cc:374] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:53.117107 12233 raft_consensus.cc:729] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bfbb60590ed4b67bfde22ab0445a176, State: Initialized, Role: FOLLOWER
I20260807 03:02:53.117763 12233 consensus_queue.cc:260] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.117941 12234 ts_tablet_manager.cc:1428] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260807 03:02:53.118446 12232 raft_consensus.cc:348] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.118873 12232 raft_consensus.cc:374] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:53.118963 12232 raft_consensus.cc:729] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6235eb5c57124efbb9c2eeaac23a2293, State: Initialized, Role: FOLLOWER
I20260807 03:02:53.119558 12233 ts_tablet_manager.cc:1428] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260807 03:02:53.119583 12232 consensus_queue.cc:260] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.120743 12016 heartbeater.cc:503] Master 127.5.122.62:44635 was elected leader, sending a full tablet report...
I20260807 03:02:53.122504 12232 ts_tablet_manager.cc:1428] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260807 03:02:53.159926 12243 raft_consensus.cc:482] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:53.160238 12243 raft_consensus.cc:504] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.162037 12243 leader_election.cc:288] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bfbb60590ed4b67bfde22ab0445a176 (127.5.122.1:33755), 6235eb5c57124efbb9c2eeaac23a2293 (127.5.122.2:39909)
I20260807 03:02:53.168040 12227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.162637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:39281 (local address 127.5.122.2:39909)
0807 03:02:53.162827 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.162831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.163457 (+ 626us) server_negotiation.cc:407] Connection header received
0807 03:02:53.163535 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.163540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.163616 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.163708 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:53.164080 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.164793 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.166427 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.167187 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.167307 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.167398 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.167675 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.167805 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 03:02:53.168062 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.162570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39909 (local address 127.5.122.3:39281)
0807 03:02:53.163199 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:53.163223 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:53.163372 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:53.163734 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:53.163738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:53.163760 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:53.163943 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.163952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.164915 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.164921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.166256 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.166268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.167359 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.167365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.167528 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.167553 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 03:02:53.167632 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 03:02:53.167757 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":42}
I20260807 03:02:53.168759 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.162512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:60853 (local address 127.5.122.1:33755)
0807 03:02:53.162719 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.162725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.163595 (+ 870us) server_negotiation.cc:407] Connection header received
0807 03:02:53.163923 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.163928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.164006 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.164094 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:02:53.164717 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.165448 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.167115 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.167873 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.167989 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.168085 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.168437 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.168550 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 03:02:53.169185 11955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9944a43b2a4946990955244a401871" candidate_uuid: "fba352f9002541c8b0bc35d2e9523a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6235eb5c57124efbb9c2eeaac23a2293" is_pre_election: true
I20260807 03:02:53.169425 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.162790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:33755 (local address 127.5.122.3:60853)
0807 03:02:53.163505 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:53.163522 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:53.163708 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:53.164253 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:53.164258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:53.164280 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:02:53.164558 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.164565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.165586 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.165590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.166967 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:53.166979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.168078 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.168084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:53.168251 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.168286 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 03:02:53.168939 (+ 653us) client_negotiation.cc:240] Negotiation successful
0807 03:02:53.169063 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":2,"thread_start_us":428,"threads_started":1}
I20260807 03:02:53.169747 11955 raft_consensus.cc:2454] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fba352f9002541c8b0bc35d2e9523a22 in term 0.
I20260807 03:02:53.170437 12041 leader_election.cc:302] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6235eb5c57124efbb9c2eeaac23a2293, fba352f9002541c8b0bc35d2e9523a22; no voters:
I20260807 03:02:53.170404 11780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9944a43b2a4946990955244a401871" candidate_uuid: "fba352f9002541c8b0bc35d2e9523a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bfbb60590ed4b67bfde22ab0445a176" is_pre_election: true
I20260807 03:02:53.170789 12243 raft_consensus.cc:2781] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:53.170897 11780 raft_consensus.cc:2454] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fba352f9002541c8b0bc35d2e9523a22 in term 0.
I20260807 03:02:53.170966 12243 raft_consensus.cc:482] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:53.171062 12243 raft_consensus.cc:3037] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:53.173237 12243 raft_consensus.cc:504] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.173748 12243 leader_election.cc:288] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [CANDIDATE]: Term 1 election: Requested vote from peers 4bfbb60590ed4b67bfde22ab0445a176 (127.5.122.1:33755), 6235eb5c57124efbb9c2eeaac23a2293 (127.5.122.2:39909)
I20260807 03:02:53.174126 11780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9944a43b2a4946990955244a401871" candidate_uuid: "fba352f9002541c8b0bc35d2e9523a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bfbb60590ed4b67bfde22ab0445a176"
I20260807 03:02:53.174201 11955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9944a43b2a4946990955244a401871" candidate_uuid: "fba352f9002541c8b0bc35d2e9523a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6235eb5c57124efbb9c2eeaac23a2293"
I20260807 03:02:53.174291 11780 raft_consensus.cc:3037] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:53.174440 11955 raft_consensus.cc:3037] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:53.176457 11780 raft_consensus.cc:2454] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fba352f9002541c8b0bc35d2e9523a22 in term 1.
I20260807 03:02:53.176680 11955 raft_consensus.cc:2454] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fba352f9002541c8b0bc35d2e9523a22 in term 1.
I20260807 03:02:53.176827 12043 leader_election.cc:302] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bfbb60590ed4b67bfde22ab0445a176, fba352f9002541c8b0bc35d2e9523a22; no voters:
I20260807 03:02:53.177131 12243 raft_consensus.cc:2781] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:53.177537 12243 raft_consensus.cc:686] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 1 LEADER]: Becoming Leader. State: Replica: fba352f9002541c8b0bc35d2e9523a22, State: Running, Role: LEADER
I20260807 03:02:53.178005 12243 consensus_queue.cc:237] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } }
I20260807 03:02:53.182235 11558 catalog_manager.cc:5495] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 reported cstate change: term changed from 0 to 1, leader changed from <none> to fba352f9002541c8b0bc35d2e9523a22 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "fba352f9002541c8b0bc35d2e9523a22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fba352f9002541c8b0bc35d2e9523a22" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 } health_report { overall_health: UNKNOWN } } }
I20260807 03:02:53.199013 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.193785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.5.122.3:36605)
0807 03:02:53.193995 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.194001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.194083 (+ 82us) server_negotiation.cc:407] Connection header received
0807 03:02:53.194291 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.194294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.194428 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.194553 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:02:53.194918 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.195625 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.196939 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.197343 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.197448 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.197896 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:02:53.198458 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:02:53.198585 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.198685 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.198763 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:02:53.206964 11955 raft_consensus.cc:1261] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 1 FOLLOWER]: Refusing update from remote peer fba352f9002541c8b0bc35d2e9523a22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:53.206964 11780 raft_consensus.cc:1261] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 1 FOLLOWER]: Refusing update from remote peer fba352f9002541c8b0bc35d2e9523a22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:53.208123 12243 consensus_queue.cc:1035] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bfbb60590ed4b67bfde22ab0445a176" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 33755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:53.208500 12249 consensus_queue.cc:1035] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6235eb5c57124efbb9c2eeaac23a2293" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:53.219705 12252 mvcc.cc:205] Tried to move back new op lower bound from 7315749983045390336 to 7315749982940774400. Current Snapshot: MvccSnapshot[applied={T|T < 7315749983045390336}]
I20260807 03:02:53.233423 11558 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:53.233680 11558 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:53.237133 11558 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72a4730013fe4c3a87d0a32a42933c3b: Sending DeleteTablet for 3 replicas of tablet dc9944a43b2a4946990955244a401871
I20260807 03:02:53.238381 11747 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9944a43b2a4946990955244a401871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:53 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:36488
I20260807 03:02:53.238411 11926 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9944a43b2a4946990955244a401871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:53 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47020
I20260807 03:02:53.239236 12269 tablet_replica.cc:330] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: stopping tablet replica
I20260807 03:02:53.239496 12106 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9944a43b2a4946990955244a401871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:53 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:37384
I20260807 03:02:53.239807 12269 raft_consensus.cc:2229] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:53.240433 12269 raft_consensus.cc:2258] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:53.241020 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 11649
I20260807 03:02:53.242746 12269 ts_tablet_manager.cc:1905] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:53.250062 12269 ts_tablet_manager.cc:1918] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:53.250365 12269 log.cc:1196] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/dc9944a43b2a4946990955244a401871
I20260807 03:02:53.250964 12269 ts_tablet_manager.cc:1939] T dc9944a43b2a4946990955244a401871 P 6235eb5c57124efbb9c2eeaac23a2293: Deleting consensus metadata
I20260807 03:02:53.253908 11538 catalog_manager.cc:4841] TS 6235eb5c57124efbb9c2eeaac23a2293 (127.5.122.2:39909): tablet dc9944a43b2a4946990955244a401871 (table test-table [id=422c1a3754414797915e5d2eb4cd880b]) successfully deleted
I20260807 03:02:53.254402 12271 tablet_replica.cc:330] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: stopping tablet replica
I20260807 03:02:53.254849 12271 raft_consensus.cc:2229] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:53.255352 12271 raft_consensus.cc:2258] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:53.257658 12271 ts_tablet_manager.cc:1905] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:53.260568 12272 tablet_replica.cc:330] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: stopping tablet replica
I20260807 03:02:53.260921 12272 raft_consensus.cc:2229] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:53.261453 12272 raft_consensus.cc:2258] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:53.263957 12272 ts_tablet_manager.cc:1905] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:53.265249 12271 ts_tablet_manager.cc:1918] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:53.265419 12271 log.cc:1196] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/dc9944a43b2a4946990955244a401871
I20260807 03:02:53.265962 12271 ts_tablet_manager.cc:1939] T dc9944a43b2a4946990955244a401871 P 4bfbb60590ed4b67bfde22ab0445a176: Deleting consensus metadata
Aug 07 03:02:53 dist-test-slave-fgb3 krb5kdc[11500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:02:53.267747 11540 catalog_manager.cc:4841] TS 4bfbb60590ed4b67bfde22ab0445a176 (127.5.122.1:33755): tablet dc9944a43b2a4946990955244a401871 (table test-table [id=422c1a3754414797915e5d2eb4cd880b]) successfully deleted
I20260807 03:02:53.271349 12272 ts_tablet_manager.cc:1918] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:53.271545 12272 log.cc:1196] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/dc9944a43b2a4946990955244a401871
I20260807 03:02:53.272086 12272 ts_tablet_manager.cc:1939] T dc9944a43b2a4946990955244a401871 P fba352f9002541c8b0bc35d2e9523a22: Deleting consensus metadata
I20260807 03:02:53.273931 11540 catalog_manager.cc:4841] TS fba352f9002541c8b0bc35d2e9523a22 (127.5.122.3:36605): tablet dc9944a43b2a4946990955244a401871 (table test-table [id=422c1a3754414797915e5d2eb4cd880b]) successfully deleted
I20260807 03:02:53.275661 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.242763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36490 (local address 127.5.122.1:33755)
0807 03:02:53.242924 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.242930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.254871 (+ 11941us) server_negotiation.cc:407] Connection header received
0807 03:02:53.255241 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.255247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.255393 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.255474 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:53.257368 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.258632 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.262183 (+ 3551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.262884 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.263023 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.268576 (+ 5553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:53.268590 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:53.268606 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:53.268648 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:53.272463 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.273384 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.273393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.273396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.273556 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.273987 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.273993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.273997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.274459 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:53.274766 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.275070 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.275286 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 03:02:53.276491 11833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:53.402158 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 11649
I20260807 03:02:53.423527 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 11848
Aug 07 03:02:53 dist-test-slave-fgb3 krb5kdc[11500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:02:53.440624 12227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.424364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47034 (local address 127.5.122.2:39909)
0807 03:02:53.424528 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.424532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.424550 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:02:53.424679 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.424683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.424788 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.424870 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:53.426509 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.427296 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.429020 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.429413 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.429503 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.434669 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:53.434687 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:53.434703 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:53.434750 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:53.437709 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.438540 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.438548 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.438551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.438701 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.439071 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.439074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.439076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.439482 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:53.439729 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.440032 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.440266 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 03:02:53.441383 12010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:53.545552 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 11848
I20260807 03:02:53.566638 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 12022
Aug 07 03:02:53 dist-test-slave-fgb3 krb5kdc[11500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:02:53.582005 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:53.567412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.5.122.3:36605)
0807 03:02:53.567590 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:53.567596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:53.567620 (+ 24us) server_negotiation.cc:407] Connection header received
0807 03:02:53.567776 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:53.567782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:53.567860 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:53.567943 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:53.569187 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.570014 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.571358 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:53.571834 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:53.571946 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:53.576184 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:53.576197 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:53.576212 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:02:53.576243 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:53.579230 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.579869 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.579880 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.579883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.580101 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:02:53.580486 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:02:53.580490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:02:53.580492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:02:53.580865 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:53.581194 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:53.581430 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 03:02:53.581644 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:02:53.582770 12192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:53.704941 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 12022
I20260807 03:02:53.727964 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 11516
I20260807 03:02:53.729130 11600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:53.837031 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 11516
2026-08-07T03:02:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4193 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T03:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:53Z Disabled control of system clock
I20260807 03:02:53.911862 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44887
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:44887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 03:02:54.224658 12285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:54.224928 12285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:54.225034 12285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:54.232168 12285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:54.232282 12285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:54.232321 12285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:54.232399 12285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:54.232456 12285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:02:54.241549 12285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:44887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:54.242873 12285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:54.244694 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:54.255381 12299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:54.255466 12300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:54.255414 12302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:54.256636 12285 server_base.cc:1034] running on GCE node
I20260807 03:02:54.257344 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:54.258677 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:54.259891 12285 hybrid_clock.cc:648] HybridClock initialized: now 1786071774259842 us; error 41 us; skew 500 ppm
I20260807 03:02:54.260227 12285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:54.262383 12285 webserver.cc:460] Webserver started at http://127.5.122.62:46873/ using document root <none> and password file <none>
I20260807 03:02:54.263036 12285 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:54.263141 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:54.263448 12285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:54.266199 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "2d28b8a56d2a441a88d1196e0ce5f595"
format_stamp: "Formatted at 2026-08-07 03:02:54 on dist-test-slave-fgb3"
server_key: "dd25e01a4a3ba70d50ca3390a0ec544e"
server_key_iv: "00586e896f67423f84daa318554b78df"
server_key_version: "encryptionkey@0"
I20260807 03:02:54.267007 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "2d28b8a56d2a441a88d1196e0ce5f595"
format_stamp: "Formatted at 2026-08-07 03:02:54 on dist-test-slave-fgb3"
server_key: "dd25e01a4a3ba70d50ca3390a0ec544e"
server_key_iv: "00586e896f67423f84daa318554b78df"
server_key_version: "encryptionkey@0"
I20260807 03:02:54.271903 12285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 03:02:54.275316 12311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:54.276657 12285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:02:54.276839 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "2d28b8a56d2a441a88d1196e0ce5f595"
format_stamp: "Formatted at 2026-08-07 03:02:54 on dist-test-slave-fgb3"
server_key: "dd25e01a4a3ba70d50ca3390a0ec544e"
server_key_iv: "00586e896f67423f84daa318554b78df"
server_key_version: "encryptionkey@0"
I20260807 03:02:54.277057 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:54.282945 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:54.283720 12285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:54.284037 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:54.302018 12285 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:44887
I20260807 03:02:54.302043 12384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:44887 every 8 connection(s)
I20260807 03:02:54.303416 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:54.307057 12386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:54.310037 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 12285
I20260807 03:02:54.310238 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:54.311069 5608 external_mini_cluster.cc:1428] Setting key f70fca3060118d277ae019ba8ac67e64
I20260807 03:02:54.319128 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:54.313183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52728 (local address 127.5.122.62:44887)
0807 03:02:54.314133 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:54.314143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:54.314183 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:02:54.315809 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:54.315830 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:54.316236 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:54.316555 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:54.317619 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:54.317625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:54.317637 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:54.317651 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:54.317713 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:54.317737 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:54.317896 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:54.318010 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 03:02:54.318236 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":355,"threads_started":1}
I20260807 03:02:54.319679 12386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595: Bootstrap starting.
I20260807 03:02:54.322861 12386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:54.323773 12386 log.cc:824] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:54.326658 12386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595: No bootstrap required, opened a new log
I20260807 03:02:54.334267 12386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44887 } }
I20260807 03:02:54.334695 12386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:54.334805 12386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d28b8a56d2a441a88d1196e0ce5f595, State: Initialized, Role: FOLLOWER
I20260807 03:02:54.335408 12386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44887 } }
I20260807 03:02:54.335666 12386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:54.335841 12386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:54.336038 12386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:54.338392 12386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44887 } }
I20260807 03:02:54.338901 12386 leader_election.cc:302] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d28b8a56d2a441a88d1196e0ce5f595; no voters:
I20260807 03:02:54.339483 12386 leader_election.cc:288] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:54.339586 12394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:54.340348 12394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [term 1 LEADER]: Becoming Leader. State: Replica: 2d28b8a56d2a441a88d1196e0ce5f595, State: Running, Role: LEADER
I20260807 03:02:54.340883 12394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44887 } }
I20260807 03:02:54.341601 12386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:54.343578 12395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44887 } } }
I20260807 03:02:54.343921 12395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:54.343536 12397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d28b8a56d2a441a88d1196e0ce5f595. Latest consensus state: current_term: 1 leader_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d28b8a56d2a441a88d1196e0ce5f595" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44887 } } }
I20260807 03:02:54.344794 12397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:54.347589 12407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:54.352382 12407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:54.361016 12407 catalog_manager.cc:1347] Generated new cluster ID: 9951f6307b5840caac5d6b6b8e9a841a
I20260807 03:02:54.361140 12407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:54.378502 12407 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:54.378901 12407 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:54.385155 12407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595: Generated new TSK 0
I20260807 03:02:54.385854 12407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:02:54.389897 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44887
--builtin_ntp_servers=127.5.122.20:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 03:02:54.736187 12421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:54.736444 12421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:54.736570 12421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:54.743700 12421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:54.743788 12421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:54.743925 12421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:02:54.752833 12421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:54.753994 12421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:54.755693 12421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:54.767761 12429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:54.767794 12430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:54.767710 12433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:54.768633 12421 server_base.cc:1034] running on GCE node
I20260807 03:02:54.769182 12421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:54.770067 12421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:54.771282 12421 hybrid_clock.cc:648] HybridClock initialized: now 1786071774771231 us; error 29 us; skew 500 ppm
I20260807 03:02:54.771622 12421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:54.773772 12421 webserver.cc:460] Webserver started at http://127.5.122.1:45511/ using document root <none> and password file <none>
I20260807 03:02:54.774482 12421 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:54.774613 12421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:54.774942 12421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:54.777854 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "e77c5c108c254d4aa3a417ba5f5b513d"
format_stamp: "Formatted at 2026-08-07 03:02:54 on dist-test-slave-fgb3"
server_key: "8da9c9ae89f6c33a67a90b0775b392e1"
server_key_iv: "887532041a05ab15371922a33735aaf5"
server_key_version: "encryptionkey@0"
I20260807 03:02:54.778697 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "e77c5c108c254d4aa3a417ba5f5b513d"
format_stamp: "Formatted at 2026-08-07 03:02:54 on dist-test-slave-fgb3"
server_key: "8da9c9ae89f6c33a67a90b0775b392e1"
server_key_iv: "887532041a05ab15371922a33735aaf5"
server_key_version: "encryptionkey@0"
I20260807 03:02:54.783739 12421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:02:54.787026 12439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:54.788244 12421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:54.788419 12421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "e77c5c108c254d4aa3a417ba5f5b513d"
format_stamp: "Formatted at 2026-08-07 03:02:54 on dist-test-slave-fgb3"
server_key: "8da9c9ae89f6c33a67a90b0775b392e1"
server_key_iv: "887532041a05ab15371922a33735aaf5"
server_key_version: "encryptionkey@0"
I20260807 03:02:54.788625 12421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:54.793998 12421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:54.794793 12421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:54.795028 12421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:54.795919 12421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:54.797257 12421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:54.797351 12421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:54.797454 12421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:54.797526 12421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:54.827948 12421 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:43681
I20260807 03:02:54.827999 12592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:43681 every 8 connection(s)
I20260807 03:02:54.829140 12421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:02:54.833810 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 12421
I20260807 03:02:54.833976 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:02:54.834617 5608 external_mini_cluster.cc:1428] Setting key a783e384a3dce9104d83212d5f99b8cb
I20260807 03:02:54.836596 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:54.831438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:36833 (local address 127.5.122.62:44887)
0807 03:02:54.831853 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:54.831858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:54.832838 (+ 980us) server_negotiation.cc:407] Connection header received
0807 03:02:54.834521 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:54.834527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:54.834591 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:54.834687 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:54.835865 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:54.835872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:54.835876 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:54.835892 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:54.835934 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:54.835946 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:54.836044 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:54.836228 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 03:02:54.836293 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":215,"threads_started":1}
I20260807 03:02:54.837105 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44887
--builtin_ntp_servers=127.5.122.20:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 03:02:54.837436 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:54.831878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44887 (local address 127.5.122.1:36833)
0807 03:02:54.832700 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:54.832733 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:54.834244 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:54.834916 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:54.834923 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:54.835001 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:54.835591 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:54.835593 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:54.835627 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:54.835704 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:54.836029 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:54.836038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:54.836046 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 03:02:54.836301 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 03:02:54.836526 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":249,"threads_started":1}
I20260807 03:02:54.838873 12593 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44887
I20260807 03:02:54.839119 12593 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:54.839632 12593 heartbeater.cc:511] Master 127.5.122.62:44887 requested a full tablet report, sending...
I20260807 03:02:54.841631 12338 ts_manager.cc:194] Registered new tserver with Master: e77c5c108c254d4aa3a417ba5f5b513d (127.5.122.1:43681)
W20260807 03:02:55.152060 12602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:55.152321 12602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:55.152449 12602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:55.159503 12602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:55.159593 12602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:55.159703 12602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:02:55.168380 12602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:55.169545 12602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:55.171249 12602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:55.183813 12608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:55.183844 12609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:55.183951 12612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:55.184887 12602 server_base.cc:1034] running on GCE node
I20260807 03:02:55.185428 12602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:55.186303 12602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:55.187549 12602 hybrid_clock.cc:648] HybridClock initialized: now 1786071775187507 us; error 35 us; skew 500 ppm
I20260807 03:02:55.187865 12602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:55.190030 12602 webserver.cc:460] Webserver started at http://127.5.122.2:43477/ using document root <none> and password file <none>
I20260807 03:02:55.190707 12602 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:55.190819 12602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:55.191114 12602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:55.193858 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "f3733f1ec4f64bc18ea7a8f409961e6c"
format_stamp: "Formatted at 2026-08-07 03:02:55 on dist-test-slave-fgb3"
server_key: "e7e468c655cfef8b4922a532c3fd0891"
server_key_iv: "a48a54e0661a9406bab4216a24a9a817"
server_key_version: "encryptionkey@0"
I20260807 03:02:55.194681 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "f3733f1ec4f64bc18ea7a8f409961e6c"
format_stamp: "Formatted at 2026-08-07 03:02:55 on dist-test-slave-fgb3"
server_key: "e7e468c655cfef8b4922a532c3fd0891"
server_key_iv: "a48a54e0661a9406bab4216a24a9a817"
server_key_version: "encryptionkey@0"
I20260807 03:02:55.199659 12602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 03:02:55.203074 12619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:55.204406 12602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:55.204567 12602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "f3733f1ec4f64bc18ea7a8f409961e6c"
format_stamp: "Formatted at 2026-08-07 03:02:55 on dist-test-slave-fgb3"
server_key: "e7e468c655cfef8b4922a532c3fd0891"
server_key_iv: "a48a54e0661a9406bab4216a24a9a817"
server_key_version: "encryptionkey@0"
I20260807 03:02:55.204798 12602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:55.210525 12602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:55.211299 12602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:55.211549 12602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:55.212435 12602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:55.213829 12602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:55.213917 12602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:55.214030 12602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:55.214099 12602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:55.244966 12602 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:46413
I20260807 03:02:55.245034 12782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:46413 every 8 connection(s)
I20260807 03:02:55.246275 12602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:02:55.248333 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 12602
I20260807 03:02:55.248494 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:02:55.249032 5608 external_mini_cluster.cc:1428] Setting key cdce42ec7fe5c5a163088f18e9d722bb
I20260807 03:02:55.252143 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44887
--builtin_ntp_servers=127.5.122.20:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 03:02:55.254997 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.248887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:43723 (local address 127.5.122.62:44887)
0807 03:02:55.249091 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.249098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.250647 (+ 1549us) server_negotiation.cc:407] Connection header received
0807 03:02:55.252863 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.252868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.252928 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.253011 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.254422 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.254425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.254429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.254444 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.254476 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.254486 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.254585 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.254759 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.254811 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:02:55.256187 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.249468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44887 (local address 127.5.122.2:43723)
0807 03:02:55.250503 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:55.250546 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:55.252598 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:55.253263 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:55.253276 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:55.253395 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:55.254071 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:55.254075 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:55.254122 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:55.254229 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:55.254573 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:55.254582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:55.254588 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 03:02:55.254844 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 03:02:55.255093 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":333,"threads_started":1}
I20260807 03:02:55.257426 12783 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44887
I20260807 03:02:55.257694 12783 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:55.258266 12783 heartbeater.cc:511] Master 127.5.122.62:44887 requested a full tablet report, sending...
I20260807 03:02:55.259588 12338 ts_manager.cc:194] Registered new tserver with Master: f3733f1ec4f64bc18ea7a8f409961e6c (127.5.122.2:46413)
W20260807 03:02:55.570878 12786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:55.571156 12786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:55.571296 12786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:55.578644 12786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:55.578748 12786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:55.578886 12786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:02:55.588223 12786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:55.589545 12786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:55.591361 12786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:55.604303 12797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:55.604357 12799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:55.604485 12796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:55.605082 12786 server_base.cc:1034] running on GCE node
I20260807 03:02:55.605669 12786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:55.606649 12786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:55.607846 12786 hybrid_clock.cc:648] HybridClock initialized: now 1786071775607803 us; error 34 us; skew 500 ppm
I20260807 03:02:55.608191 12786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:55.614580 12786 webserver.cc:460] Webserver started at http://127.5.122.3:45097/ using document root <none> and password file <none>
I20260807 03:02:55.615265 12786 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:55.615375 12786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:55.615680 12786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:55.618594 12786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "0df4fd0e846b47dea37460f19c6029c6"
format_stamp: "Formatted at 2026-08-07 03:02:55 on dist-test-slave-fgb3"
server_key: "6bd8799e935c0634fe77bcf74c5ac592"
server_key_iv: "922c7cdee6c49e52d1771ac1ccea7aa4"
server_key_version: "encryptionkey@0"
I20260807 03:02:55.619359 12786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "0df4fd0e846b47dea37460f19c6029c6"
format_stamp: "Formatted at 2026-08-07 03:02:55 on dist-test-slave-fgb3"
server_key: "6bd8799e935c0634fe77bcf74c5ac592"
server_key_iv: "922c7cdee6c49e52d1771ac1ccea7aa4"
server_key_version: "encryptionkey@0"
I20260807 03:02:55.624287 12786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:02:55.627581 12811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:55.628842 12786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:02:55.628989 12786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "0df4fd0e846b47dea37460f19c6029c6"
format_stamp: "Formatted at 2026-08-07 03:02:55 on dist-test-slave-fgb3"
server_key: "6bd8799e935c0634fe77bcf74c5ac592"
server_key_iv: "922c7cdee6c49e52d1771ac1ccea7aa4"
server_key_version: "encryptionkey@0"
I20260807 03:02:55.629200 12786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:55.634768 12786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:55.635515 12786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:55.635761 12786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:55.636794 12786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:02:55.638212 12786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:02:55.638307 12786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:55.638482 12786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:02:55.638558 12786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:55.670146 12786 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36459
I20260807 03:02:55.670197 12974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36459 every 8 connection(s)
I20260807 03:02:55.671412 12786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:02:55.673480 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 12786
I20260807 03:02:55.673623 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:02:55.674149 5608 external_mini_cluster.cc:1428] Setting key 41f253b4b9762c1ed45d96dd6670efb8
I20260807 03:02:55.679471 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.673900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:58235 (local address 127.5.122.62:44887)
0807 03:02:55.674106 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.674112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.675621 (+ 1509us) server_negotiation.cc:407] Connection header received
0807 03:02:55.677594 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.677598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.677647 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.677724 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.678905 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.678910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.678914 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.678930 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.678964 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.678976 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.679080 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.679206 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.679257 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 03:02:55.680332 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.674465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44887 (local address 127.5.122.3:58235)
0807 03:02:55.675453 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:55.675510 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:55.677349 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:55.677900 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:55.677907 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:55.678003 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:55.678616 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:55.678618 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:55.678653 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:55.678733 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:55.679060 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:55.679065 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:55.679070 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:02:55.679262 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 03:02:55.679453 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":303,"threads_started":1}
I20260807 03:02:55.681476 12975 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44887
I20260807 03:02:55.681730 12975 heartbeater.cc:464] Registering TS with master...
I20260807 03:02:55.682260 12975 heartbeater.cc:511] Master 127.5.122.62:44887 requested a full tablet report, sending...
I20260807 03:02:55.683554 12338 ts_manager.cc:194] Registered new tserver with Master: 0df4fd0e846b47dea37460f19c6029c6 (127.5.122.3:36459)
I20260807 03:02:55.689074 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:02:55.695678 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.693292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.5.122.62:44887)
0807 03:02:55.693455 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.693459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.693965 (+ 506us) server_negotiation.cc:407] Connection header received
0807 03:02:55.694116 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.694121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.694177 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.694256 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.695092 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.695098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.695103 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.695121 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.695151 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.695162 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.695253 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.695358 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.695425 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:02:55.699271 12338 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:02:55.706077 12338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:02:55.734126 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.719191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.5.122.1:43681)
0807 03:02:55.728522 (+ 9331us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.728532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.731216 (+ 2684us) server_negotiation.cc:407] Connection header received
0807 03:02:55.731303 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.731311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.731522 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.731665 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.733235 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.733244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.733258 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.733282 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.733325 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.733339 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.733471 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.733683 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.733781 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9132,"thread_start_us":8980,"threads_started":1}
I20260807 03:02:55.734160 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.718914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:32976)
0807 03:02:55.731082 (+ 12168us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:55.731098 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:55.731197 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:55.731714 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:55.731718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:55.731740 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:55.732518 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:55.732521 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:55.732553 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:55.732642 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:55.733496 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:55.733507 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:55.733515 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 03:02:55.733681 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 03:02:55.733768 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12068,"thread_start_us":9764,"threads_started":1}
I20260807 03:02:55.735111 12989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.717923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:46413 (local address 127.0.0.1:52250)
0807 03:02:55.730168 (+ 12245us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:55.730189 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:55.730295 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:55.731412 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:55.731420 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:55.731511 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:55.732874 (+ 1363us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:55.732877 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:55.732891 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:55.732930 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:55.734429 (+ 1499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:55.734435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:55.734440 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:02:55.734565 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 03:02:55.734616 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12095,"thread_start_us":11916,"threads_started":1}
I20260807 03:02:55.735138 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.718010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.5.122.3:36459)
0807 03:02:55.731762 (+ 13752us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.731769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.731801 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:02:55.731866 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.731873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.732067 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.732208 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.734158 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.734166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.734178 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.734197 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.734250 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.734270 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.734626 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.734687 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.734777 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13593,"thread_start_us":13416,"threads_started":1}
I20260807 03:02:55.735472 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.717882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36459 (local address 127.0.0.1:60264)
0807 03:02:55.729125 (+ 11243us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:55.729185 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:55.729336 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:55.732401 (+ 3065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:55.732406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:55.732433 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:55.733901 (+ 1468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:55.733904 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:55.733920 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:55.733964 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:55.734545 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:55.734551 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:55.734556 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:02:55.734702 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 03:02:55.734755 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10933,"thread_start_us":12970,"threads_started":1}
I20260807 03:02:55.735922 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.718462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.5.122.2:46413)
0807 03:02:55.730831 (+ 12369us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.730839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.730872 (+ 33us) server_negotiation.cc:407] Connection header received
0807 03:02:55.730946 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.730954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.731202 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.733128 (+ 1926us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.734179 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.734185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.734194 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.734216 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.734261 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.734278 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.735415 (+ 1137us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.735490 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.735582 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12200,"thread_start_us":12070,"threads_started":1}
I20260807 03:02:55.738747 12879 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc05772096a461e9f78b62d34e3a2f2 (DEFAULT_TABLE table=test-table [id=844f275c560941e3b145a80d518789ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:55.738776 12502 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc05772096a461e9f78b62d34e3a2f2 (DEFAULT_TABLE table=test-table [id=844f275c560941e3b145a80d518789ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:55.739403 12690 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc05772096a461e9f78b62d34e3a2f2 (DEFAULT_TABLE table=test-table [id=844f275c560941e3b145a80d518789ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:02:55.739991 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc05772096a461e9f78b62d34e3a2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:55.740134 12502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc05772096a461e9f78b62d34e3a2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:55.740660 12690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc05772096a461e9f78b62d34e3a2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:55.752256 12996 tablet_bootstrap.cc:492] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Bootstrap starting.
I20260807 03:02:55.754658 12997 tablet_bootstrap.cc:492] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: Bootstrap starting.
I20260807 03:02:55.755434 12999 tablet_bootstrap.cc:492] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Bootstrap starting.
I20260807 03:02:55.755591 12996 tablet_bootstrap.cc:654] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:55.756896 12996 log.cc:824] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:55.758445 12997 tablet_bootstrap.cc:654] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:55.758949 12999 tablet_bootstrap.cc:654] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:55.759650 12997 log.cc:824] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:55.759861 12999 log.cc:824] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:55.760238 12996 tablet_bootstrap.cc:492] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: No bootstrap required, opened a new log
I20260807 03:02:55.760574 12996 ts_tablet_manager.cc:1397] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260807 03:02:55.762606 12997 tablet_bootstrap.cc:492] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: No bootstrap required, opened a new log
I20260807 03:02:55.762606 12999 tablet_bootstrap.cc:492] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: No bootstrap required, opened a new log
I20260807 03:02:55.762866 12999 ts_tablet_manager.cc:1397] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 03:02:55.762862 12997 ts_tablet_manager.cc:1397] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260807 03:02:55.767863 12996 raft_consensus.cc:348] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.768203 12996 raft_consensus.cc:374] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:55.768328 12996 raft_consensus.cc:729] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df4fd0e846b47dea37460f19c6029c6, State: Initialized, Role: FOLLOWER
I20260807 03:02:55.768895 12996 consensus_queue.cc:260] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.770026 12975 heartbeater.cc:503] Master 127.5.122.62:44887 was elected leader, sending a full tablet report...
I20260807 03:02:55.770705 12996 ts_tablet_manager.cc:1428] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260807 03:02:55.772917 12997 raft_consensus.cc:348] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.772917 12999 raft_consensus.cc:348] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.773332 12997 raft_consensus.cc:374] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:55.773334 12999 raft_consensus.cc:374] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:55.773437 12999 raft_consensus.cc:729] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3733f1ec4f64bc18ea7a8f409961e6c, State: Initialized, Role: FOLLOWER
I20260807 03:02:55.773437 12997 raft_consensus.cc:729] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e77c5c108c254d4aa3a417ba5f5b513d, State: Initialized, Role: FOLLOWER
I20260807 03:02:55.774016 12999 consensus_queue.cc:260] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.774024 12997 consensus_queue.cc:260] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.775133 12783 heartbeater.cc:503] Master 127.5.122.62:44887 was elected leader, sending a full tablet report...
I20260807 03:02:55.775838 12593 heartbeater.cc:503] Master 127.5.122.62:44887 was elected leader, sending a full tablet report...
I20260807 03:02:55.775857 12999 ts_tablet_manager.cc:1428] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 03:02:55.776710 12997 ts_tablet_manager.cc:1428] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 03:02:55.779294 13008 raft_consensus.cc:482] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:02:55.779512 13008 raft_consensus.cc:504] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.781116 13008 leader_election.cc:288] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0df4fd0e846b47dea37460f19c6029c6 (127.5.122.3:36459), e77c5c108c254d4aa3a417ba5f5b513d (127.5.122.1:43681)
I20260807 03:02:55.784129 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.781444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:53871 (local address 127.5.122.3:36459)
0807 03:02:55.781751 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.781755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.782220 (+ 465us) server_negotiation.cc:407] Connection header received
0807 03:02:55.782475 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.782481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.782544 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.782645 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.783573 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.783579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.783584 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.783603 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.783636 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.783647 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.783754 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.783839 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.783897 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214}
I20260807 03:02:55.784343 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.781419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36459 (local address 127.5.122.2:53871)
0807 03:02:55.782106 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:55.782134 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:55.782255 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:55.782709 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:55.782714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:55.782739 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:55.783384 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:55.783386 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:55.783403 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:55.783441 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:55.783732 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:55.783737 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:55.783742 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:02:55.783882 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 03:02:55.783925 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":350,"threads_started":1}
I20260807 03:02:55.784709 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.781458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:56197 (local address 127.5.122.1:43681)
0807 03:02:55.781618 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.781622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.782937 (+ 1315us) server_negotiation.cc:407] Connection header received
0807 03:02:55.783073 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.783078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.783118 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.783194 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.784066 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.784072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.784077 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.784095 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.784130 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.784143 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.784253 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.784471 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.784526 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:02:55.785240 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.782069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.5.122.2:56197)
0807 03:02:55.782843 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:55.782857 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:55.782960 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:55.783238 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:55.783242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:55.783265 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:02:55.783863 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:02:55.783866 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:02:55.783882 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:02:55.783923 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:02:55.784310 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:02:55.784315 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:02:55.784319 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 03:02:55.784918 (+ 599us) client_negotiation.cc:240] Negotiation successful
0807 03:02:55.784968 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":443,"threads_started":1}
I20260807 03:02:55.785197 12910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc05772096a461e9f78b62d34e3a2f2" candidate_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df4fd0e846b47dea37460f19c6029c6" is_pre_election: true
I20260807 03:02:55.785634 12910 raft_consensus.cc:2454] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3733f1ec4f64bc18ea7a8f409961e6c in term 0.
I20260807 03:02:55.786026 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc05772096a461e9f78b62d34e3a2f2" candidate_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" is_pre_election: true
I20260807 03:02:55.786252 12623 leader_election.cc:302] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df4fd0e846b47dea37460f19c6029c6, f3733f1ec4f64bc18ea7a8f409961e6c; no voters:
I20260807 03:02:55.786532 12536 raft_consensus.cc:2454] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3733f1ec4f64bc18ea7a8f409961e6c in term 0.
I20260807 03:02:55.786661 13008 raft_consensus.cc:2781] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:02:55.786808 13008 raft_consensus.cc:482] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:02:55.786900 13008 raft_consensus.cc:3037] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:55.789029 13008 raft_consensus.cc:504] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.789551 13008 leader_election.cc:288] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [CANDIDATE]: Term 1 election: Requested vote from peers 0df4fd0e846b47dea37460f19c6029c6 (127.5.122.3:36459), e77c5c108c254d4aa3a417ba5f5b513d (127.5.122.1:43681)
I20260807 03:02:55.789894 12910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc05772096a461e9f78b62d34e3a2f2" candidate_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df4fd0e846b47dea37460f19c6029c6"
I20260807 03:02:55.789906 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc05772096a461e9f78b62d34e3a2f2" candidate_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e77c5c108c254d4aa3a417ba5f5b513d"
I20260807 03:02:55.790045 12910 raft_consensus.cc:3037] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:55.790038 12536 raft_consensus.cc:3037] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:55.792615 12536 raft_consensus.cc:2454] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3733f1ec4f64bc18ea7a8f409961e6c in term 1.
I20260807 03:02:55.792686 12910 raft_consensus.cc:2454] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3733f1ec4f64bc18ea7a8f409961e6c in term 1.
I20260807 03:02:55.792963 12622 leader_election.cc:302] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e77c5c108c254d4aa3a417ba5f5b513d, f3733f1ec4f64bc18ea7a8f409961e6c; no voters:
I20260807 03:02:55.793249 13008 raft_consensus.cc:2781] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:55.793627 13008 raft_consensus.cc:686] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 1 LEADER]: Becoming Leader. State: Replica: f3733f1ec4f64bc18ea7a8f409961e6c, State: Running, Role: LEADER
I20260807 03:02:55.794036 13008 consensus_queue.cc:237] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } }
I20260807 03:02:55.797827 12338 catalog_manager.cc:5495] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c reported cstate change: term changed from 0 to 1, leader changed from <none> to f3733f1ec4f64bc18ea7a8f409961e6c (127.5.122.2). New cstate: current_term: 1 leader_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3733f1ec4f64bc18ea7a8f409961e6c" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 } health_report { overall_health: UNKNOWN } } }
I20260807 03:02:55.814198 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.812624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.5.122.2:46413)
0807 03:02:55.812754 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.812758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.812814 (+ 56us) server_negotiation.cc:407] Connection header received
0807 03:02:55.812980 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.812983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.813049 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.813124 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.813799 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.813803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.813806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.813820 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.813849 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.813859 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.813945 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.813993 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.814032 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 03:02:55.821192 12536 raft_consensus.cc:1261] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d [term 1 FOLLOWER]: Refusing update from remote peer f3733f1ec4f64bc18ea7a8f409961e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:55.821226 12910 raft_consensus.cc:1261] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 1 FOLLOWER]: Refusing update from remote peer f3733f1ec4f64bc18ea7a8f409961e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:02:55.822024 13011 consensus_queue.cc:1035] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e77c5c108c254d4aa3a417ba5f5b513d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:02:55.822389 13008 consensus_queue.cc:1035] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0df4fd0e846b47dea37460f19c6029c6" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 03:02:55.831099 12594 tablet.cc:2369] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:02:55.839242 13016 mvcc.cc:205] Tried to move back new op lower bound from 7315749993755705344 to 7315749993656008704. Current Snapshot: MvccSnapshot[applied={T|T < 7315749993755705344}]
I20260807 03:02:55.843876 13018 mvcc.cc:205] Tried to move back new op lower bound from 7315749993755705344 to 7315749993656008704. Current Snapshot: MvccSnapshot[applied={T|T < 7315749993755705344}]
I20260807 03:02:55.855973 12337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:55.856262 12337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:02:55.860798 12337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d28b8a56d2a441a88d1196e0ce5f595: Sending DeleteTablet for 3 replicas of tablet 5bc05772096a461e9f78b62d34e3a2f2
I20260807 03:02:55.862231 12879 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc05772096a461e9f78b62d34e3a2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:55 UTC) from {username='slave'} at 127.0.0.1:60264
I20260807 03:02:55.862470 12502 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc05772096a461e9f78b62d34e3a2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:55 UTC) from {username='slave'} at 127.0.0.1:32976
I20260807 03:02:55.862977 12690 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc05772096a461e9f78b62d34e3a2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:02:55 UTC) from {username='slave'} at 127.0.0.1:52250
I20260807 03:02:55.865023 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 12421
I20260807 03:02:55.874459 13030 tablet_replica.cc:330] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: stopping tablet replica
I20260807 03:02:55.875022 13030 raft_consensus.cc:2229] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:02:55.875515 13030 raft_consensus.cc:2258] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:55.885334 13033 tablet_replica.cc:330] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: stopping tablet replica
I20260807 03:02:55.887256 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:55.866670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.5.122.1:43681)
0807 03:02:55.866852 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:55.866859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:55.884686 (+ 17827us) server_negotiation.cc:407] Connection header received
0807 03:02:55.884874 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:55.884880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:55.885006 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:55.885116 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:55.886416 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:55.886423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:55.886429 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:55.886452 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:55.886494 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:55.886510 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:55.886610 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:55.886837 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 03:02:55.886904 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 03:02:55.888042 13030 ts_tablet_manager.cc:1905] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:55.888500 12587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:55.891058 13033 raft_consensus.cc:2229] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:02:55.891815 13033 raft_consensus.cc:2258] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:02:55.896178 13033 ts_tablet_manager.cc:1905] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:02:55.896924 13030 ts_tablet_manager.cc:1918] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:55.897138 13030 log.cc:1196] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/5bc05772096a461e9f78b62d34e3a2f2
I20260807 03:02:55.897724 13030 ts_tablet_manager.cc:1939] T 5bc05772096a461e9f78b62d34e3a2f2 P 0df4fd0e846b47dea37460f19c6029c6: Deleting consensus metadata
I20260807 03:02:55.899632 12317 catalog_manager.cc:4841] TS 0df4fd0e846b47dea37460f19c6029c6 (127.5.122.3:36459): tablet 5bc05772096a461e9f78b62d34e3a2f2 (table test-table [id=844f275c560941e3b145a80d518789ee]) successfully deleted
I20260807 03:02:55.903638 13033 ts_tablet_manager.cc:1918] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:02:55.903784 13033 log.cc:1196] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/5bc05772096a461e9f78b62d34e3a2f2
I20260807 03:02:55.904201 13033 ts_tablet_manager.cc:1939] T 5bc05772096a461e9f78b62d34e3a2f2 P f3733f1ec4f64bc18ea7a8f409961e6c: Deleting consensus metadata
I20260807 03:02:55.905490 12312 catalog_manager.cc:4841] TS f3733f1ec4f64bc18ea7a8f409961e6c (127.5.122.2:46413): tablet 5bc05772096a461e9f78b62d34e3a2f2 (table test-table [id=844f275c560941e3b145a80d518789ee]) successfully deleted
I20260807 03:02:56.004316 13032 tablet_replica.cc:330] T 5bc05772096a461e9f78b62d34e3a2f2 P e77c5c108c254d4aa3a417ba5f5b513d: stopping tablet replica
I20260807 03:02:56.004591 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 12421
I20260807 03:02:56.027000 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 12602
W20260807 03:02:56.027012 12314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.1:43681 (error 108)
I20260807 03:02:56.029654 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.029122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:32986)
0807 03:02:56.029304 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.029521 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 03:02:56.029907 12314 catalog_manager.cc:4584] TS e77c5c108c254d4aa3a417ba5f5b513d (127.5.122.1:43681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bc05772096a461e9f78b62d34e3a2f2: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
I20260807 03:02:56.030447 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.027740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.5.122.2:46413)
0807 03:02:56.027949 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:56.027956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:56.028220 (+ 264us) server_negotiation.cc:407] Connection header received
0807 03:02:56.028321 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:56.028330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:56.028411 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:56.028526 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:56.029517 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:56.029523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:56.029528 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:56.029548 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:56.029586 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:56.029602 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:56.029743 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:56.030097 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 03:02:56.030161 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 03:02:56.031065 12776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:56.083582 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.083008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:33000)
0807 03:02:56.083251 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.083382 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 03:02:56.084705 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.084339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:33016)
0807 03:02:56.084486 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.084567 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 03:02:56.142328 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 12602
I20260807 03:02:56.145550 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.145005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:33022)
0807 03:02:56.145225 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.145346 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 03:02:56.146639 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.146208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:33026)
0807 03:02:56.146394 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.146486 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 03:02:56.166441 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 12786
I20260807 03:02:56.169032 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.167145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.5.122.3:36459)
0807 03:02:56.167355 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:56.167359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:56.167378 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:02:56.167478 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:56.167482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:56.167560 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:56.167641 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:56.168443 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:56.168447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:56.168451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:56.168466 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:56.168499 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:56.168511 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:56.168614 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:56.168684 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 03:02:56.168735 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 03:02:56.169611 12968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:56.227686 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.227074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:33030)
0807 03:02:56.227346 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.227479 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 03:02:56.228844 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.228381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43681 (local address 127.0.0.1:33046)
0807 03:02:56.228620 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.228703 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 03:02:56.256323 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 12786
I20260807 03:02:56.277786 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 12285
I20260807 03:02:56.279000 12379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:56.383833 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 12285
2026-08-07T03:02:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2544 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 03:02:56.428858 5608 mini_oidc.cc:152] Starting JWKS server
I20260807 03:02:56.431713 5608 webserver.cc:460] Webserver started at https://127.0.0.1:35309/ using document root <none> and password file <none>
I20260807 03:02:56.432036 5608 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 03:02:56.432585 5608 webserver.cc:460] Webserver started at http://127.0.0.1:45421/ using document root <none> and password file <none>
2026-08-07T03:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:56Z Disabled control of system clock
I20260807 03:02:56.461719 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:34533
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:34533
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35309/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/testchainca.cert with env {}
W20260807 03:02:56.772994 13050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:56.773265 13050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:56.773383 13050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:56.780619 13050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:56.780730 13050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:56.780799 13050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:56.780879 13050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:56.780939 13050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 03:02:56.781025 13050 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 03:02:56.781071 13050 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35309/jwks.json
I20260807 03:02:56.791167 13050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:34533
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:34533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35309/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:56.792424 13050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:56.793704 13050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:56.804946 13056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:56.804867 13057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:56.804919 13060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:56.805894 13050 server_base.cc:1034] running on GCE node
I20260807 03:02:56.806681 13050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:56.807961 13050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:56.809170 13050 hybrid_clock.cc:648] HybridClock initialized: now 1786071776809128 us; error 35 us; skew 500 ppm
I20260807 03:02:56.809510 13050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:56.811700 13050 webserver.cc:460] Webserver started at http://127.5.122.62:41063/ using document root <none> and password file <none>
I20260807 03:02:56.812350 13050 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:56.812456 13050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:56.812744 13050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:56.815593 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec"
format_stamp: "Formatted at 2026-08-07 03:02:56 on dist-test-slave-fgb3"
server_key: "9195c45591f1c1cb196ece2de212b818"
server_key_iv: "7fa9ef9ad6d53c2758542be9807abe87"
server_key_version: "encryptionkey@0"
I20260807 03:02:56.816349 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec"
format_stamp: "Formatted at 2026-08-07 03:02:56 on dist-test-slave-fgb3"
server_key: "9195c45591f1c1cb196ece2de212b818"
server_key_iv: "7fa9ef9ad6d53c2758542be9807abe87"
server_key_version: "encryptionkey@0"
I20260807 03:02:56.821116 13050 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 03:02:56.824256 13066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:56.825423 13050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:02:56.825587 13050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec"
format_stamp: "Formatted at 2026-08-07 03:02:56 on dist-test-slave-fgb3"
server_key: "9195c45591f1c1cb196ece2de212b818"
server_key_iv: "7fa9ef9ad6d53c2758542be9807abe87"
server_key_version: "encryptionkey@0"
I20260807 03:02:56.825771 13050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:56.839557 13050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:56.840260 13050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:56.840543 13050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:56.857254 13050 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:34533
I20260807 03:02:56.857283 13140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:34533 every 8 connection(s)
I20260807 03:02:56.858505 13050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:56.858990 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 13050
I20260807 03:02:56.859182 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:56.859902 5608 external_mini_cluster.cc:1428] Setting key bbbfee7fbbdbebe13344e407c8389232
I20260807 03:02:56.862637 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:56.872344 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.861663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.5.122.62:34533)
0807 03:02:56.862618 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:56.862639 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:56.862680 (+ 41us) server_negotiation.cc:407] Connection header received
0807 03:02:56.864267 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:56.864288 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:56.864687 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:56.865046 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:56.865650 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:56.867022 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:56.868330 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:56.868938 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:56.869095 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:56.870198 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:56.870204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:56.870219 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:56.870787 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:56.870863 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:56.870886 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:56.870988 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:56.871201 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 03:02:56.871488 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":305,"threads_started":1}
I20260807 03:02:56.872747 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec: Bootstrap starting.
I20260807 03:02:56.875998 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:02:56.876232 13141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:56.877157 13141 log.cc:824] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:56.879973 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec: No bootstrap required, opened a new log
I20260807 03:02:56.887221 13141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34533 } }
I20260807 03:02:56.887567 13141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:56.887660 13141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f4d3fb28fc4f1c9e44775fe7f23aec, State: Initialized, Role: FOLLOWER
I20260807 03:02:56.888221 13141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34533 } }
I20260807 03:02:56.888454 13141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:56.888582 13141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:56.888738 13141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:56.890848 13141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34533 } }
I20260807 03:02:56.891312 13141 leader_election.cc:302] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64f4d3fb28fc4f1c9e44775fe7f23aec; no voters:
I20260807 03:02:56.891841 13141 leader_election.cc:288] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:56.891987 13147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:56.892613 13147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [term 1 LEADER]: Becoming Leader. State: Replica: 64f4d3fb28fc4f1c9e44775fe7f23aec, State: Running, Role: LEADER
I20260807 03:02:56.893164 13147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34533 } }
I20260807 03:02:56.893826 13141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:56.895627 13148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34533 } } }
I20260807 03:02:56.895993 13148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:56.896365 13149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64f4d3fb28fc4f1c9e44775fe7f23aec. Latest consensus state: current_term: 1 leader_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f4d3fb28fc4f1c9e44775fe7f23aec" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34533 } } }
I20260807 03:02:56.896632 13149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:56.899189 13154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:56.903383 13154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:56.911623 13154 catalog_manager.cc:1347] Generated new cluster ID: 7752f4aed06e4273bb29dfa12379639d
I20260807 03:02:56.911744 13154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:56.939934 13154 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:56.941224 13154 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:56.948143 13154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64f4d3fb28fc4f1c9e44775fe7f23aec: Generated new TSK 0
I20260807 03:02:56.948918 13154 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 03:02:56.998618 13181 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 03:02:56.998867 13181 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 03:02:56.999908 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:56.971549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.5.122.62:34533)
0807 03:02:56.971714 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:56.971721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:56.972228 (+ 507us) server_negotiation.cc:407] Connection header received
0807 03:02:56.972615 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:56.972619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:56.972697 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:56.972835 (+ 138us) server_negotiation.cc:226] Negotiated authn=JWT
0807 03:02:56.973347 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:56.974064 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:56.975538 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:56.975990 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:56.976109 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:56.976673 (+ 564us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 03:02:56.999187 (+ 22514us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 03:02:56.999578 (+ 391us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45544: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 03:02:57.000087 13144 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45544: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 03:02:57.001946 13180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:02:56.971498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:34533 (local address 127.0.0.1:45544)
0807 03:02:56.972114 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:56.972130 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:56.972326 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:56.972876 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:56.972881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:56.972894 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 03:02:56.973198 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:56.973208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:56.974203 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:56.974208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:56.975397 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:56.975409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:56.976178 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:56.976186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:56.976341 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:56.976402 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 03:02:56.999511 (+ 23109us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 03:02:56.999871 (+ 360us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.62:34533: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":302,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.122.62:34533: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 03:02:57.003499 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 13050
I20260807 03:02:57.004666 13134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:57.096573 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 13050
2026-08-07T03:02:57Z chronyd exiting
I20260807 03:02:57.152695 5608 test_util.cc:175] -----------------------------------------------
I20260807 03:02:57.152854 5608 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786071733698599-5608-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (729 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 03:02:57.156615 5608 mini_oidc.cc:152] Starting JWKS server
I20260807 03:02:57.158474 5608 webserver.cc:460] Webserver started at https://127.0.0.1:45673/ using document root <none> and password file <none>
I20260807 03:02:57.158685 5608 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 03:02:57.159298 5608 webserver.cc:460] Webserver started at http://127.0.0.1:36093/ using document root <none> and password file <none>
2026-08-07T03:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:57Z Disabled control of system clock
I20260807 03:02:57.189739 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:43187
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:46277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:43187
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45673/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/testchainca.cert with env {}
W20260807 03:02:57.502154 13192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:57.502476 13192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:57.502596 13192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:57.509785 13192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:57.509896 13192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:57.509941 13192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:57.510021 13192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:57.510073 13192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 03:02:57.510159 13192 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 03:02:57.510210 13192 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45673/jwks.json
I20260807 03:02:57.519953 13192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:46277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:43187
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45673/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:57.521220 13192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:57.522559 13192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:57.533510 13198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:57.533473 13202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:57.533553 13200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:57.534691 13192 server_base.cc:1034] running on GCE node
I20260807 03:02:57.535477 13192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:57.536785 13192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:57.537977 13192 hybrid_clock.cc:648] HybridClock initialized: now 1786071777537939 us; error 33 us; skew 500 ppm
I20260807 03:02:57.538295 13192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:57.540386 13192 webserver.cc:460] Webserver started at http://127.5.122.62:36399/ using document root <none> and password file <none>
I20260807 03:02:57.541018 13192 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:57.541105 13192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:57.541369 13192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:57.544206 13192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "63f297753c99412391ea4199d86fca2a"
format_stamp: "Formatted at 2026-08-07 03:02:57 on dist-test-slave-fgb3"
server_key: "61bdfd1bd2a28100b88b51cefa936375"
server_key_iv: "a59fd5ac0926d1920823eec5f8674010"
server_key_version: "encryptionkey@0"
I20260807 03:02:57.544962 13192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "63f297753c99412391ea4199d86fca2a"
format_stamp: "Formatted at 2026-08-07 03:02:57 on dist-test-slave-fgb3"
server_key: "61bdfd1bd2a28100b88b51cefa936375"
server_key_iv: "a59fd5ac0926d1920823eec5f8674010"
server_key_version: "encryptionkey@0"
I20260807 03:02:57.549945 13192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 03:02:57.553193 13210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:57.554515 13192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:02:57.554656 13192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "63f297753c99412391ea4199d86fca2a"
format_stamp: "Formatted at 2026-08-07 03:02:57 on dist-test-slave-fgb3"
server_key: "61bdfd1bd2a28100b88b51cefa936375"
server_key_iv: "a59fd5ac0926d1920823eec5f8674010"
server_key_version: "encryptionkey@0"
I20260807 03:02:57.554828 13192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:57.571770 13192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:57.572527 13192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:57.572782 13192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:57.589447 13192 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:43187
I20260807 03:02:57.589480 13270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:43187 every 8 connection(s)
I20260807 03:02:57.590718 13192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:57.594184 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:57.598052 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 13192
I20260807 03:02:57.598245 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:57.599092 5608 external_mini_cluster.cc:1428] Setting key 4b97d731f888ab2a92a17be4d0b9495f
I20260807 03:02:57.605165 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a: Bootstrap starting.
I20260807 03:02:57.608085 13271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:57.609144 13271 log.cc:824] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:57.612326 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a: No bootstrap required, opened a new log
I20260807 03:02:57.612534 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:57.600982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.5.122.62:43187)
0807 03:02:57.601917 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:57.601927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:57.601970 (+ 43us) server_negotiation.cc:407] Connection header received
0807 03:02:57.603932 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:57.603954 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:57.604357 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:57.604691 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:57.605211 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:57.606580 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:57.607786 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:57.608578 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:57.608741 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:57.609953 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:57.609962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:57.609975 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:57.610588 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:57.610686 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:57.610718 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:57.610837 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:57.611022 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 03:02:57.611339 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":351,"threads_started":1}
I20260807 03:02:57.616016 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:02:57.620760 13271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f297753c99412391ea4199d86fca2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 43187 } }
I20260807 03:02:57.621080 13271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:57.621160 13271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f297753c99412391ea4199d86fca2a, State: Initialized, Role: FOLLOWER
I20260807 03:02:57.621616 13271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f297753c99412391ea4199d86fca2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 43187 } }
I20260807 03:02:57.621825 13271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:57.621922 13271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:57.622041 13271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:57.624136 13271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f297753c99412391ea4199d86fca2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 43187 } }
I20260807 03:02:57.624560 13271 leader_election.cc:302] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63f297753c99412391ea4199d86fca2a; no voters:
I20260807 03:02:57.625082 13271 leader_election.cc:288] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:57.625213 13277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:57.625873 13277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [term 1 LEADER]: Becoming Leader. State: Replica: 63f297753c99412391ea4199d86fca2a, State: Running, Role: LEADER
I20260807 03:02:57.626549 13277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f297753c99412391ea4199d86fca2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 43187 } }
I20260807 03:02:57.627053 13271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:57.629173 13278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63f297753c99412391ea4199d86fca2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f297753c99412391ea4199d86fca2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 43187 } } }
I20260807 03:02:57.629599 13278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:57.629199 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63f297753c99412391ea4199d86fca2a. Latest consensus state: current_term: 1 leader_uuid: "63f297753c99412391ea4199d86fca2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f297753c99412391ea4199d86fca2a" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 43187 } } }
I20260807 03:02:57.630455 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:57.631781 13288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:57.637008 13288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:57.644918 13288 catalog_manager.cc:1347] Generated new cluster ID: c4de5cfc024f4cb9ae9cacf7a134e723
I20260807 03:02:57.645017 13288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:57.663846 13288 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:57.665658 13288 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:57.673322 13288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63f297753c99412391ea4199d86fca2a: Generated new TSK 0
I20260807 03:02:57.674503 13288 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 03:02:57.710651 13311 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 03:02:57.710774 13311 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 03:02:57.712000 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:57.692276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.5.122.62:43187)
0807 03:02:57.692467 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:57.692477 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:57.692811 (+ 334us) server_negotiation.cc:407] Connection header received
0807 03:02:57.693023 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:57.693029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:57.693114 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:57.693297 (+ 183us) server_negotiation.cc:226] Negotiated authn=JWT
0807 03:02:57.693697 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:57.694872 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:57.696179 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:57.696893 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:57.697013 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:57.697490 (+ 477us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 03:02:57.711314 (+ 13824us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 03:02:57.711693 (+ 379us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 03:02:57.712039 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:02:57.692110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:43187 (local address 127.0.0.1:57430)
0807 03:02:57.692714 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:57.692729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:57.692849 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:57.693293 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:57.693296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:57.693305 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 03:02:57.693546 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:57.693554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:57.694992 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:57.694996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:57.696046 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:57.696058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:57.697055 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:57.697060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:57.697195 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:57.697227 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 03:02:57.711558 (+ 14331us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 03:02:57.711753 (+ 195us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.62:43187: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":328,"threads_started":1}
W20260807 03:02:57.712127 13274 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260807 03:02:57.713280 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 13192
I20260807 03:02:57.714373 13266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:57.805335 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 13192
2026-08-07T03:02:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (706 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 03:02:57.863596 5608 mini_oidc.cc:152] Starting JWKS server
I20260807 03:02:57.865435 5608 webserver.cc:460] Webserver started at https://127.0.0.1:41125/ using document root <none> and password file <none>
I20260807 03:02:57.865630 5608 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 03:02:57.866199 5608 webserver.cc:460] Webserver started at http://127.0.0.1:35519/ using document root <none> and password file <none>
2026-08-07T03:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:57Z Disabled control of system clock
I20260807 03:02:57.896484 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39819
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39819
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41125/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 03:02:58.223645 13321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:58.223963 13321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:58.224113 13321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:58.231494 13321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:58.231617 13321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:58.231686 13321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:58.231777 13321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:58.231850 13321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 03:02:58.231945 13321 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 03:02:58.232009 13321 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41125/jwks.json
I20260807 03:02:58.241470 13321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41125/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:58.242877 13321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:58.244760 13321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:58.256258 13327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:58.256274 13328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:58.256629 13331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:58.257263 13321 server_base.cc:1034] running on GCE node
I20260807 03:02:58.257932 13321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:58.259267 13321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:58.260461 13321 hybrid_clock.cc:648] HybridClock initialized: now 1786071778260423 us; error 31 us; skew 500 ppm
I20260807 03:02:58.260761 13321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:58.262835 13321 webserver.cc:460] Webserver started at http://127.5.122.62:36947/ using document root <none> and password file <none>
I20260807 03:02:58.263460 13321 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:58.263545 13321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:58.263810 13321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:58.266590 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "d36d527ce5c0493bbe66336be45eac57"
format_stamp: "Formatted at 2026-08-07 03:02:58 on dist-test-slave-fgb3"
server_key: "021c81810a15650ce2bcf2f65a7b4254"
server_key_iv: "6b58eff131016947acbd5d2287185d58"
server_key_version: "encryptionkey@0"
I20260807 03:02:58.267338 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "d36d527ce5c0493bbe66336be45eac57"
format_stamp: "Formatted at 2026-08-07 03:02:58 on dist-test-slave-fgb3"
server_key: "021c81810a15650ce2bcf2f65a7b4254"
server_key_iv: "6b58eff131016947acbd5d2287185d58"
server_key_version: "encryptionkey@0"
I20260807 03:02:58.272073 13321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:02:58.275172 13340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:58.276366 13321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:02:58.276504 13321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "d36d527ce5c0493bbe66336be45eac57"
format_stamp: "Formatted at 2026-08-07 03:02:58 on dist-test-slave-fgb3"
server_key: "021c81810a15650ce2bcf2f65a7b4254"
server_key_iv: "6b58eff131016947acbd5d2287185d58"
server_key_version: "encryptionkey@0"
I20260807 03:02:58.276680 13321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:58.293794 13321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:58.294529 13321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:58.294776 13321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:58.311434 13321 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:39819
I20260807 03:02:58.311451 13410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:39819 every 8 connection(s)
I20260807 03:02:58.312678 13321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:58.315622 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 13321
I20260807 03:02:58.315850 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:58.316541 13412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:58.316666 5608 external_mini_cluster.cc:1428] Setting key 2836abab203f4f26c896d8dc7051687e
I20260807 03:02:58.329319 13416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:58.318451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.5.122.62:39819)
0807 03:02:58.319412 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:58.319422 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:58.319462 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:02:58.321078 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:58.321098 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:58.321504 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:58.321802 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:58.322378 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:58.323661 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:58.325365 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:58.325981 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:58.326111 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:58.327285 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:58.327293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:58.327305 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:58.327816 (+ 511us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:58.327895 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:58.327920 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:58.328001 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:58.328197 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 03:02:58.328467 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":366,"threads_started":1}
I20260807 03:02:58.329605 13412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57: Bootstrap starting.
I20260807 03:02:58.332949 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:02:58.333076 13412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:58.334205 13412 log.cc:824] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:58.336990 13412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57: No bootstrap required, opened a new log
I20260807 03:02:58.343844 13412 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d527ce5c0493bbe66336be45eac57" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39819 } }
I20260807 03:02:58.344163 13412 raft_consensus.cc:374] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:58.344234 13412 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36d527ce5c0493bbe66336be45eac57, State: Initialized, Role: FOLLOWER
I20260807 03:02:58.344746 13412 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d527ce5c0493bbe66336be45eac57" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39819 } }
I20260807 03:02:58.344962 13412 raft_consensus.cc:388] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:58.345057 13412 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:58.345178 13412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:58.347411 13412 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d527ce5c0493bbe66336be45eac57" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39819 } }
I20260807 03:02:58.347869 13412 leader_election.cc:302] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d36d527ce5c0493bbe66336be45eac57; no voters:
I20260807 03:02:58.348390 13412 leader_election.cc:288] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:58.348519 13418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:58.349119 13418 raft_consensus.cc:686] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [term 1 LEADER]: Becoming Leader. State: Replica: d36d527ce5c0493bbe66336be45eac57, State: Running, Role: LEADER
I20260807 03:02:58.349632 13418 consensus_queue.cc:237] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d527ce5c0493bbe66336be45eac57" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39819 } }
I20260807 03:02:58.350371 13412 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:58.352448 13419 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d36d527ce5c0493bbe66336be45eac57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d527ce5c0493bbe66336be45eac57" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39819 } } }
I20260807 03:02:58.352733 13419 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:58.353226 13420 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36d527ce5c0493bbe66336be45eac57. Latest consensus state: current_term: 1 leader_uuid: "d36d527ce5c0493bbe66336be45eac57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d527ce5c0493bbe66336be45eac57" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39819 } } }
I20260807 03:02:58.353644 13420 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57 [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:58.355111 13423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:58.360226 13423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:58.368960 13423 catalog_manager.cc:1347] Generated new cluster ID: 7d650f5b3f194f5c9bbce3d26aa2dcb5
I20260807 03:02:58.369071 13423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:58.396690 13423 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:58.398090 13423 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:58.413465 13423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d36d527ce5c0493bbe66336be45eac57: Generated new TSK 0
I20260807 03:02:58.414222 13423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:02:58.438563 13416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:58.431553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.5.122.62:39819)
0807 03:02:58.431714 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:58.431721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:58.432126 (+ 405us) server_negotiation.cc:407] Connection header received
0807 03:02:58.432352 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:58.432357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:58.432431 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:58.432557 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0807 03:02:58.433016 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:58.433737 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:58.435089 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:58.435585 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:58.435691 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:58.436175 (+ 484us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 03:02:58.437941 (+ 1766us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41125/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 03:02:58.438241 (+ 300us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51220: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41125/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 03:02:58.438777 13454 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:02:58.431476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39819 (local address 127.0.0.1:51220)
0807 03:02:58.432024 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:58.432043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:58.432177 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:58.432598 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:58.432603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:58.432615 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0807 03:02:58.432875 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:58.432885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:58.433877 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:58.433880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:58.434958 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:58.434969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:58.435778 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:58.435784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:58.435909 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:58.435941 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 03:02:58.438135 (+ 2194us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41125/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 03:02:58.438387 (+ 252us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.62:39819: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41125/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":313,"threads_started":1}
W20260807 03:02:58.439006 13416 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51220: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41125/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260807 03:02:58.439810 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 13321
I20260807 03:02:58.440837 13405 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 03:02:58.450258 13455 webserver.cc:533] Webserver: sslize failed:
E20260807 03:02:58.450368 13455 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 03:02:58.533588 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 13321
2026-08-07T03:02:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (728 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T03:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:02:58Z Disabled control of system clock
I20260807 03:02:58.623927 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42185
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:37713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:42185
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/testchainca.cert with env {}
W20260807 03:02:58.934744 13461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:02:58.935045 13461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:02:58.935174 13461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:02:58.942368 13461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:02:58.942478 13461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:02:58.942541 13461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:02:58.942616 13461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:02:58.942670 13461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 03:02:58.942754 13461 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 03:02:58.942798 13461 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 03:02:58.952538 13461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:42185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:02:58.953794 13461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:02:58.955051 13461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:02:58.965975 13469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:58.965934 13474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:02:58.966408 13468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:02:58.966874 13461 server_base.cc:1034] running on GCE node
I20260807 03:02:58.967592 13461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:02:58.968822 13461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:02:58.970014 13461 hybrid_clock.cc:648] HybridClock initialized: now 1786071778969969 us; error 36 us; skew 500 ppm
I20260807 03:02:58.970347 13461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:02:58.972393 13461 webserver.cc:460] Webserver started at http://127.5.122.62:41171/ using document root <none> and password file <none>
I20260807 03:02:58.973024 13461 fs_manager.cc:359] Metadata directory not provided
I20260807 03:02:58.973124 13461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:02:58.973423 13461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:02:58.976163 13461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c"
format_stamp: "Formatted at 2026-08-07 03:02:58 on dist-test-slave-fgb3"
server_key: "2636949e1df93de97f66000f398ef491"
server_key_iv: "d7f74ebee49532fd6ae081915a741810"
server_key_version: "encryptionkey@0"
I20260807 03:02:58.976923 13461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c"
format_stamp: "Formatted at 2026-08-07 03:02:58 on dist-test-slave-fgb3"
server_key: "2636949e1df93de97f66000f398ef491"
server_key_iv: "d7f74ebee49532fd6ae081915a741810"
server_key_version: "encryptionkey@0"
I20260807 03:02:58.981714 13461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 03:02:58.984968 13488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:02:58.986176 13461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:02:58.986380 13461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c"
format_stamp: "Formatted at 2026-08-07 03:02:58 on dist-test-slave-fgb3"
server_key: "2636949e1df93de97f66000f398ef491"
server_key_iv: "d7f74ebee49532fd6ae081915a741810"
server_key_version: "encryptionkey@0"
I20260807 03:02:58.986579 13461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:02:59.009986 13461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:02:59.010826 13461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:02:59.011098 13461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:02:59.027729 13461 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:42185
I20260807 03:02:59.027748 13559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:42185 every 8 connection(s)
I20260807 03:02:59.028993 13461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:02:59.032395 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 13461
I20260807 03:02:59.032567 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:02:59.032617 13560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:02:59.033215 5608 external_mini_cluster.cc:1428] Setting key 0c1cbeb437d317c3554c2a2513a4debb
I20260807 03:02:59.046478 13560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c: Bootstrap starting.
I20260807 03:02:59.048517 13564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:59.035062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.5.122.62:42185)
0807 03:02:59.036106 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:59.036119 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:59.036175 (+ 56us) server_negotiation.cc:407] Connection header received
0807 03:02:59.038881 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:59.038909 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:59.039481 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:59.039899 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:02:59.040417 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:59.042222 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:59.043477 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:59.044525 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:59.044693 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:59.045714 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:02:59.045725 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:02:59.045742 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:02:59.046444 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:02:59.046546 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:02:59.046573 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:02:59.046688 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 03:02:59.046932 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 03:02:59.047253 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":305,"threads_started":1}
I20260807 03:02:59.049978 13560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c: Neither blocks nor log segments found. Creating new log.
I20260807 03:02:59.051049 13560 log.cc:824] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c: Log is configured to *not* fsync() on all Append() calls
I20260807 03:02:59.052086 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:02:59.053769 13560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c: No bootstrap required, opened a new log
I20260807 03:02:59.061647 13560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42185 } }
I20260807 03:02:59.061983 13560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:02:59.062077 13560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b8c62bd3d0940f7b1aa50d9e8eaf49c, State: Initialized, Role: FOLLOWER
I20260807 03:02:59.062610 13560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42185 } }
I20260807 03:02:59.062812 13560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:02:59.062951 13560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:02:59.063082 13560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:02:59.065187 13560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42185 } }
I20260807 03:02:59.065657 13560 leader_election.cc:302] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b8c62bd3d0940f7b1aa50d9e8eaf49c; no voters:
I20260807 03:02:59.066341 13560 leader_election.cc:288] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:02:59.066459 13567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:02:59.067157 13567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [term 1 LEADER]: Becoming Leader. State: Replica: 6b8c62bd3d0940f7b1aa50d9e8eaf49c, State: Running, Role: LEADER
I20260807 03:02:59.067698 13567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42185 } }
I20260807 03:02:59.068401 13560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:02:59.070673 13568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42185 } } }
I20260807 03:02:59.071108 13568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:59.071321 13571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b8c62bd3d0940f7b1aa50d9e8eaf49c. Latest consensus state: current_term: 1 leader_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8c62bd3d0940f7b1aa50d9e8eaf49c" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42185 } } }
I20260807 03:02:59.071679 13571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c [sys.catalog]: This master's current role is: LEADER
I20260807 03:02:59.073060 13581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:02:59.078033 13581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:02:59.085645 13581 catalog_manager.cc:1347] Generated new cluster ID: 9fc4f661b065494d8a0e896de9bf353a
I20260807 03:02:59.085752 13581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:02:59.109411 13581 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:02:59.111251 13581 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:02:59.126876 13581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b8c62bd3d0940f7b1aa50d9e8eaf49c: Generated new TSK 0
I20260807 03:02:59.127933 13581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:02:59.604033 13564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:02:59.140328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.5.122.62:42185)
0807 03:02:59.140526 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 03:02:59.140532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:02:59.141008 (+ 476us) server_negotiation.cc:407] Connection header received
0807 03:02:59.141220 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:02:59.141226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:02:59.141313 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:02:59.141491 (+ 178us) server_negotiation.cc:226] Negotiated authn=JWT
0807 03:02:59.141926 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:59.143075 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:59.144497 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:02:59.145173 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:02:59.145296 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:59.145807 (+ 511us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 03:02:59.603475 (+457668us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 03:02:59.603798 (+ 323us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41504: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 03:02:59.604151 13564 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41504: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 03:02:59.604395 13604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:02:59.140295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42185 (local address 127.0.0.1:41504)
0807 03:02:59.140914 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 03:02:59.140931 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:02:59.141052 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:02:59.141483 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:02:59.141486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:02:59.141495 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 03:02:59.141795 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:59.141803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:59.143240 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:59.143244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:59.144354 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:02:59.144366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:02:59.145366 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:02:59.145371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:02:59.145497 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:02:59.145532 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 03:02:59.603760 (+458228us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 03:02:59.604020 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.62:42185: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":325,"threads_started":1}
I20260807 03:02:59.605414 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 13461
I20260807 03:02:59.606596 13552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:02:59.705247 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 13461
2026-08-07T03:02:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1155 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (409 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (398 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:03:00 dist-test-slave-fgb3 krb5kdc[13626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:03:00 dist-test-slave-fgb3 krb5kdc[13626](info): set up 2 sockets
Aug 07 03:03:00 dist-test-slave-fgb3 krb5kdc[13626](info): commencing operation
krb5kdc: starting...
W20260807 03:03:01.699626 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:03:01 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:03:01.915736 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:34899
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:34899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:02.229615 13642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:02.229892 13642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:02.229981 13642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:02.237028 13642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:02.237128 13642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:02.237179 13642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:02.237236 13642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:02.237278 13642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:02.246786 13642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:34899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:34899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:02.247949 13642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:02.249158 13642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:02.262523 13647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:02.262590 13652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:02.262609 13649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:02.263398 13642 server_base.cc:1034] running on GCE node
I20260807 03:03:02.264169 13642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:02.265455 13642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:02.266636 13642 hybrid_clock.cc:648] HybridClock initialized: now 1786071782266583 us; error 47 us; skew 500 ppm
I20260807 03:03:02.266965 13642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:02 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071782, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:02.272656 13642 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:03:02.274040 13642 webserver.cc:460] Webserver started at http://127.5.122.62:37053/ using document root <none> and password file <none>
I20260807 03:03:02.274693 13642 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:02.274801 13642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:02.275068 13642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:02.277784 13642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "3e4991875e084fb99d9b0bc077722c18"
format_stamp: "Formatted at 2026-08-07 03:03:02 on dist-test-slave-fgb3"
server_key: "360f36211b740abe7e463bbc4bb57cd5"
server_key_iv: "d9abf405cdc783c13705b0027f9ce5c1"
server_key_version: "encryptionkey@0"
I20260807 03:03:02.278540 13642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "3e4991875e084fb99d9b0bc077722c18"
format_stamp: "Formatted at 2026-08-07 03:03:02 on dist-test-slave-fgb3"
server_key: "360f36211b740abe7e463bbc4bb57cd5"
server_key_iv: "d9abf405cdc783c13705b0027f9ce5c1"
server_key_version: "encryptionkey@0"
I20260807 03:03:02.283215 13642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 03:03:02.286307 13659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:02.287551 13642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:03:02.287710 13642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "3e4991875e084fb99d9b0bc077722c18"
format_stamp: "Formatted at 2026-08-07 03:03:02 on dist-test-slave-fgb3"
server_key: "360f36211b740abe7e463bbc4bb57cd5"
server_key_iv: "d9abf405cdc783c13705b0027f9ce5c1"
server_key_version: "encryptionkey@0"
I20260807 03:03:02.287885 13642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:02.303589 13642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:02.307543 13642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:02.307792 13642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:02.324605 13642 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:34899
I20260807 03:03:02.324643 13727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:34899 every 8 connection(s)
I20260807 03:03:02.325950 13642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:02.329495 13729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:02.334527 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 13642
I20260807 03:03:02.334733 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:03:02.335505 5608 external_mini_cluster.cc:1428] Setting key 1c251c0b315e2094546c1196619f56ff
I20260807 03:03:02.340742 13729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18: Bootstrap starting.
I20260807 03:03:02.344329 13729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:02.345569 13729 log.cc:824] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:02.348892 13729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18: No bootstrap required, opened a new log
Aug 07 03:03:02 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:02.357549 13729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4991875e084fb99d9b0bc077722c18" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34899 } }
I20260807 03:03:02.357934 13729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:02.358047 13729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e4991875e084fb99d9b0bc077722c18, State: Initialized, Role: FOLLOWER
I20260807 03:03:02.358403 13732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:02.337611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.5.122.62:34899)
0807 03:03:02.338449 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:02.338460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:02.338504 (+ 44us) server_negotiation.cc:407] Connection header received
0807 03:03:02.340196 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:02.340219 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:02.340595 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:02.340951 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:02.342659 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:02.343935 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:02.345646 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:02.346575 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:02.346784 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:02.351279 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:02.351287 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:02.351304 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:02.351341 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:02.354390 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:02.355133 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:02.355140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:02.355146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:02.355310 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:02.355698 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:02.355701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:02.355703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:02.356231 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:02.356438 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:02.356916 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 03:03:02.357266 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":354,"threads_started":1}
I20260807 03:03:02.358723 13729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4991875e084fb99d9b0bc077722c18" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34899 } }
I20260807 03:03:02.359017 13729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:02.359155 13729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:02.359323 13729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:02.361701 13729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4991875e084fb99d9b0bc077722c18" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34899 } }
I20260807 03:03:02.362171 13729 leader_election.cc:302] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e4991875e084fb99d9b0bc077722c18; no voters:
I20260807 03:03:02.362726 13729 leader_election.cc:288] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:03:02.362835 13734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:02.363445 13734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [term 1 LEADER]: Becoming Leader. State: Replica: 3e4991875e084fb99d9b0bc077722c18, State: Running, Role: LEADER
I20260807 03:03:02.364013 13734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4991875e084fb99d9b0bc077722c18" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34899 } }
I20260807 03:03:02.364789 13729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:02.366905 13737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e4991875e084fb99d9b0bc077722c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4991875e084fb99d9b0bc077722c18" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34899 } } }
I20260807 03:03:02.366878 13738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e4991875e084fb99d9b0bc077722c18. Latest consensus state: current_term: 1 leader_uuid: "3e4991875e084fb99d9b0bc077722c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4991875e084fb99d9b0bc077722c18" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 34899 } } }
I20260807 03:03:02.367372 13737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:02.367478 13738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:02.368628 13744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:02.373360 13744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:02.382834 13744 catalog_manager.cc:1347] Generated new cluster ID: 979c797a2e0e4e3aac5ab96666586468
I20260807 03:03:02.382937 13744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:02.404903 13744 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:03:02.406744 13744 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:02.416481 13744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e4991875e084fb99d9b0bc077722c18: Generated new TSK 0
I20260807 03:03:02.417457 13744 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:03:02.467890 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:34899
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:02.784853 13765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:02.785120 13765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:02.785254 13765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:02.792549 13765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:02.792659 13765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:02.792804 13765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:02.802003 13765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:34899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:02.803174 13765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:02.804415 13765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:02.816843 13772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:02.816798 13773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:02.816941 13775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:02.817484 13765 server_base.cc:1034] running on GCE node
I20260807 03:03:02.818048 13765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:02.818974 13765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:02.820156 13765 hybrid_clock.cc:648] HybridClock initialized: now 1786071782820094 us; error 55 us; skew 500 ppm
I20260807 03:03:02.820442 13765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:02 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071782, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:02.823443 13765 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:03:02.824749 13765 webserver.cc:460] Webserver started at http://127.5.122.1:37049/ using document root <none> and password file <none>
I20260807 03:03:02.825362 13765 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:02.825485 13765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:02.825773 13765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:02.828589 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "95d0fac383e84ac184880f891b400246"
format_stamp: "Formatted at 2026-08-07 03:03:02 on dist-test-slave-fgb3"
server_key: "35f9f31d64d14f030cb93fbd4184f71a"
server_key_iv: "86497a6f6e28e0418d6f1fff63a5e514"
server_key_version: "encryptionkey@0"
I20260807 03:03:02.829324 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "95d0fac383e84ac184880f891b400246"
format_stamp: "Formatted at 2026-08-07 03:03:02 on dist-test-slave-fgb3"
server_key: "35f9f31d64d14f030cb93fbd4184f71a"
server_key_iv: "86497a6f6e28e0418d6f1fff63a5e514"
server_key_version: "encryptionkey@0"
I20260807 03:03:02.834163 13765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 03:03:02.837513 13784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:02.838851 13765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:02.838991 13765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "95d0fac383e84ac184880f891b400246"
format_stamp: "Formatted at 2026-08-07 03:03:02 on dist-test-slave-fgb3"
server_key: "35f9f31d64d14f030cb93fbd4184f71a"
server_key_iv: "86497a6f6e28e0418d6f1fff63a5e514"
server_key_version: "encryptionkey@0"
I20260807 03:03:02.839185 13765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:02.851926 13765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:02.855799 13765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:02.856076 13765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:02.857013 13765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:02.858640 13765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:02.858721 13765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:02.858837 13765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:02.858894 13765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:02.889083 13765 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:38375
I20260807 03:03:02.889129 13935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:38375 every 8 connection(s)
I20260807 03:03:02.890291 13765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:02.900147 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 13765
I20260807 03:03:02.900345 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:03:02.900972 5608 external_mini_cluster.cc:1428] Setting key 1fd3d9374efb6529269315976baedd30
Aug 07 03:03:02 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071782, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:02.908929 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:02.892844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:42829 (local address 127.5.122.62:34899)
0807 03:03:02.893257 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:02.893261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:02.894148 (+ 887us) server_negotiation.cc:407] Connection header received
0807 03:03:02.895934 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:02.895940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:02.896020 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:02.896113 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:02.897532 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:02.898294 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:02.899806 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:02.900208 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:02.900324 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:02.903705 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:02.903710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:02.903713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:02.903743 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:02.906079 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:02.906778 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:02.906784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:02.906787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:02.906904 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:02.907312 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:02.907316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:02.907318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:02.907574 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:02.907700 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:02.908346 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 03:03:02.908557 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":215,"threads_started":1}
I20260807 03:03:02.909803 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:02.893253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:34899 (local address 127.5.122.1:42829)
0807 03:03:02.893993 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:02.894037 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:02.895668 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:02.896345 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:02.896353 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:02.896842 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:02.897371 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:02.897388 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:02.898470 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:02.898474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:02.899663 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:02.899675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:02.900418 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:02.900426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:02.900640 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:02.901255 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:02.901278 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:02.903501 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:02.906213 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:02.906222 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:02.906237 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:02.906671 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:02.907004 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:02.907009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:02.907013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:02.907194 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:02.907692 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:02.907699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:02.908007 (+ 308us) client_negotiation.cc:769] Sending connection context
0807 03:03:02.908220 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 03:03:02.908715 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":309,"threads_started":1}
I20260807 03:03:02.911290 13937 heartbeater.cc:347] Connected to a master server at 127.5.122.62:34899
I20260807 03:03:02.911579 13937 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:02.912257 13937 heartbeater.cc:511] Master 127.5.122.62:34899 requested a full tablet report, sending...
I20260807 03:03:02.914481 13681 ts_manager.cc:194] Registered new tserver with Master: 95d0fac383e84ac184880f891b400246 (127.5.122.1:38375)
I20260807 03:03:02.916429 13681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:42829
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:03:02.949062 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:34899
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:03.267350 13944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:03.267606 13944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:03.267737 13944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:03.274894 13944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:03.275015 13944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:03.275164 13944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:03.284698 13944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:34899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:03.285924 13944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:03.287217 13944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:03.299620 13953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:03.299669 13958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:03.299697 13955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:03.300725 13944 server_base.cc:1034] running on GCE node
I20260807 03:03:03.301261 13944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:03.302357 13944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:03.303589 13944 hybrid_clock.cc:648] HybridClock initialized: now 1786071783303541 us; error 37 us; skew 500 ppm
I20260807 03:03:03.303882 13944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.306929 13944 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:03:03.308214 13944 webserver.cc:460] Webserver started at http://127.5.122.2:44041/ using document root <none> and password file <none>
I20260807 03:03:03.308840 13944 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:03.308952 13944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:03.309242 13944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:03.312247 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "450e55aa3cad4dac9ccd9c5d036dc490"
format_stamp: "Formatted at 2026-08-07 03:03:03 on dist-test-slave-fgb3"
server_key: "e8ed3b3fd27f6223ae9ba22fd6110f2d"
server_key_iv: "e31ed14ae936596fb9c6d59c653e9543"
server_key_version: "encryptionkey@0"
I20260807 03:03:03.313091 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "450e55aa3cad4dac9ccd9c5d036dc490"
format_stamp: "Formatted at 2026-08-07 03:03:03 on dist-test-slave-fgb3"
server_key: "e8ed3b3fd27f6223ae9ba22fd6110f2d"
server_key_iv: "e31ed14ae936596fb9c6d59c653e9543"
server_key_version: "encryptionkey@0"
I20260807 03:03:03.319043 13944 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 03:03:03.322847 13967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:03.324015 13944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:03:03.324182 13944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "450e55aa3cad4dac9ccd9c5d036dc490"
format_stamp: "Formatted at 2026-08-07 03:03:03 on dist-test-slave-fgb3"
server_key: "e8ed3b3fd27f6223ae9ba22fd6110f2d"
server_key_iv: "e31ed14ae936596fb9c6d59c653e9543"
server_key_version: "encryptionkey@0"
I20260807 03:03:03.324385 13944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:03.340312 13944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:03.344278 13944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:03.344511 13944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:03.345350 13944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:03.346643 13944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:03.346717 13944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:03.346817 13944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:03.346859 13944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:03.375113 13944 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:41541
I20260807 03:03:03.375159 14117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:41541 every 8 connection(s)
I20260807 03:03:03.376281 13944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:03.380398 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 13944
I20260807 03:03:03.380544 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:03:03.381006 5608 external_mini_cluster.cc:1428] Setting key c2c71115f855480984b18805fc3b2507
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:03.395205 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.378650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:33093 (local address 127.5.122.62:34899)
0807 03:03:03.378825 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.378829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.379955 (+ 1126us) server_negotiation.cc:407] Connection header received
0807 03:03:03.381881 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.381887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.381970 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.382063 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.383640 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.384352 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.385839 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.386261 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.386387 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.389506 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:03.389511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:03.389515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:03.389545 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:03.392371 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.393146 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.393150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.393152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.393283 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.393706 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.393709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.393711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.393942 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:03.394047 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:03.394807 (+ 760us) server_negotiation.cc:299] Negotiation successful
0807 03:03:03.394966 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 03:03:03.396450 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.379110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:34899 (local address 127.5.122.2:33093)
0807 03:03:03.379816 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:03.379851 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:03.381603 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:03.382297 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:03.382305 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:03.382966 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:03.383486 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:03.383502 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.384506 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.384510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:03.385697 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:03.385709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.386494 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.386501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:03.386721 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.387310 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:03.387331 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:03.389310 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:03.392496 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:03.392502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:03.392513 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:03.393043 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:03.393379 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:03.393384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:03.393388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:03.393619 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:03.394056 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:03.394064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:03.394457 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 03:03:03.394764 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 03:03:03.395244 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":291,"threads_started":1}
I20260807 03:03:03.397751 14119 heartbeater.cc:347] Connected to a master server at 127.5.122.62:34899
I20260807 03:03:03.398022 14119 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:03.398738 14119 heartbeater.cc:511] Master 127.5.122.62:34899 requested a full tablet report, sending...
I20260807 03:03:03.400113 13681 ts_manager.cc:194] Registered new tserver with Master: 450e55aa3cad4dac9ccd9c5d036dc490 (127.5.122.2:41541)
I20260807 03:03:03.401213 13681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:33093
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:03:03.428936 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:34899
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:03.740679 14124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:03.740921 14124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:03.741010 14124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:03.748453 14124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:03.748546 14124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:03.748658 14124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:03.757794 14124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:34899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:03.758909 14124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:03.760072 14124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:03.771950 14136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:03.772018 14132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:03.771991 14133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:03.772789 14124 server_base.cc:1034] running on GCE node
I20260807 03:03:03.773269 14124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:03.774099 14124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:03.775300 14124 hybrid_clock.cc:648] HybridClock initialized: now 1786071783775236 us; error 56 us; skew 500 ppm
I20260807 03:03:03.775592 14124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.778581 14124 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:03:03.779950 14124 webserver.cc:460] Webserver started at http://127.5.122.3:40571/ using document root <none> and password file <none>
I20260807 03:03:03.780550 14124 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:03.780663 14124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:03.780949 14124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:03.783672 14124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "2efd472e4e0b4fcd879b36d20e91d810"
format_stamp: "Formatted at 2026-08-07 03:03:03 on dist-test-slave-fgb3"
server_key: "0234141bd02d4f6bc0f86cf96fe35343"
server_key_iv: "8e480bb529821132f844735a888355ff"
server_key_version: "encryptionkey@0"
I20260807 03:03:03.784411 14124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "2efd472e4e0b4fcd879b36d20e91d810"
format_stamp: "Formatted at 2026-08-07 03:03:03 on dist-test-slave-fgb3"
server_key: "0234141bd02d4f6bc0f86cf96fe35343"
server_key_iv: "8e480bb529821132f844735a888355ff"
server_key_version: "encryptionkey@0"
I20260807 03:03:03.789068 14124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:03:03.792332 14144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:03.793540 14124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:03:03.793689 14124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "2efd472e4e0b4fcd879b36d20e91d810"
format_stamp: "Formatted at 2026-08-07 03:03:03 on dist-test-slave-fgb3"
server_key: "0234141bd02d4f6bc0f86cf96fe35343"
server_key_iv: "8e480bb529821132f844735a888355ff"
server_key_version: "encryptionkey@0"
I20260807 03:03:03.793874 14124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:03.809187 14124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:03.812847 14124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:03.813103 14124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:03.813980 14124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:03.815302 14124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:03.815380 14124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:03.815500 14124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:03.815552 14124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:03.845856 14124 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:39475
I20260807 03:03:03.845892 14289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:39475 every 8 connection(s)
I20260807 03:03:03.847152 14124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:03.848147 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 14124
I20260807 03:03:03.848351 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:03:03.848958 5608 external_mini_cluster.cc:1428] Setting key 281e3e31fa076541ead246d345c97969
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:03.866333 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.849849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53383 (local address 127.5.122.62:34899)
0807 03:03:03.850071 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.850078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.851395 (+ 1317us) server_negotiation.cc:407] Connection header received
0807 03:03:03.853184 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.853189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.853272 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.853354 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.854826 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.855561 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.857021 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.857416 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.857521 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.860868 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:03.860874 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:03.860877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:03.860908 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:03.863484 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.864136 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.864140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.864142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.864282 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.864705 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.864710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.864712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.864978 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:03.865104 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:03.865744 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 03:03:03.865968 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 03:03:03.867153 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.850445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:34899 (local address 127.5.122.3:53383)
0807 03:03:03.851249 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:03.851303 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:03.852941 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:03.853591 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:03.853600 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:03.854068 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:03.854666 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:03.854685 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.855711 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.855715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:03.856887 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:03.856899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.857622 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.857630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:03.857865 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.858589 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:03.858611 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:03.860665 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:03.863604 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:03.863612 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:03.863627 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:03.864019 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:03.864387 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:03.864392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:03.864396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:03.864576 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:03.865098 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:03.865105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:03.865408 (+ 303us) client_negotiation.cc:769] Sending connection context
0807 03:03:03.865632 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 03:03:03.866100 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":344,"threads_started":1}
I20260807 03:03:03.868333 14290 heartbeater.cc:347] Connected to a master server at 127.5.122.62:34899
I20260807 03:03:03.868603 14290 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:03.869293 14290 heartbeater.cc:511] Master 127.5.122.62:34899 requested a full tablet report, sending...
I20260807 03:03:03.870684 13681 ts_manager.cc:194] Registered new tserver with Master: 2efd472e4e0b4fcd879b36d20e91d810 (127.5.122.3:39475)
I20260807 03:03:03.871454 13681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:53383
I20260807 03:03:03.876180 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/test-admin.keytab.
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.904379 5608 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 03:03:03.909154 14301 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:03.906911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.5.122.1:38375)
0807 03:03:03.907496 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.907503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.907529 (+ 26us) server_negotiation.cc:407] Connection header received
0807 03:03:03.907588 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.907595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.907814 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.907938 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.908758 (+ 820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60668: BlockingRecv error: recv got EOF from 127.0.0.1:60668 (error 108)
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":319,"threads_started":1}
W20260807 03:03:03.909158 5608 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc' of type FILE
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.910674 5608 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.911959 5608 init.cc:295] Successfully reacquired a new kerberos TGT
I20260807 03:03:03.919142 13937 heartbeater.cc:503] Master 127.5.122.62:34899 was elected leader, sending a full tablet report...
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:03:03.926831 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.913515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.5.122.1:38375)
0807 03:03:03.913693 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.913698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.913869 (+ 171us) server_negotiation.cc:407] Connection header received
0807 03:03:03.914005 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.914009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.914066 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.914139 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.915152 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.916085 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.917222 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.917790 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.917883 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.921324 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:03.921334 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:03.921348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:03.921380 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:03.924421 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.925017 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.925023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.925026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.925140 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.925444 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.925448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.925450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.925804 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:03.926026 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:03.926250 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 03:03:03.926533 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 03:03:03.932266 14301 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:03.930863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.5.122.1:38375)
0807 03:03:03.931058 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.931064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.931159 (+ 95us) server_negotiation.cc:407] Connection header received
0807 03:03:03.931297 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.931301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.931370 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.931444 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.932047 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60690: BlockingRecv error: recv got EOF from 127.0.0.1:60690 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 03:03:03.932832 5608 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc)
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.934266 5608 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.935536 5608 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:03:03.949302 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.936810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.5.122.1:38375)
0807 03:03:03.936999 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.937004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.937193 (+ 189us) server_negotiation.cc:407] Connection header received
0807 03:03:03.937348 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.937351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.937408 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.937475 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.938338 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.939040 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.940370 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.940769 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.940856 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.944377 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:03.944383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:03.944386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:03.944419 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:03.946916 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.947543 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.947548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.947550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.947671 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.948029 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.948034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.948038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.948306 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:03.948557 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:03.948829 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 03:03:03.949025 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 03:03:03.953356 14301 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:03.951758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.5.122.1:38375)
0807 03:03:03.951943 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.951949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.952173 (+ 224us) server_negotiation.cc:407] Connection header received
0807 03:03:03.952326 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.952332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.952407 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.952484 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.953139 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60706: BlockingRecv error: recv got EOF from 127.0.0.1:60706 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 03:03:03.953506 5608 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786071733698599-5608-0/krb5kdc/krb5cc)
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.954926 5608 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:03.956123 5608 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 03:03:03 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:03:03.969481 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.957237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.5.122.1:38375)
0807 03:03:03.957436 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.957441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.957634 (+ 193us) server_negotiation.cc:407] Connection header received
0807 03:03:03.957731 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.957734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.957789 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.957856 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.958788 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.959504 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.960650 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.961046 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.961134 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.964494 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:03.964499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:03.964502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:03.964532 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:03.967346 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.967918 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.967922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.967924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.968064 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.968374 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.968377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.968379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.968636 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:03.968857 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:03.969062 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 03:03:03.969220 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 03:03:03.971036 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 13765
I20260807 03:03:03.982146 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:03.971704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.5.122.1:38375)
0807 03:03:03.971823 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:03.971827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:03.972161 (+ 334us) server_negotiation.cc:407] Connection header received
0807 03:03:03.972291 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:03.972294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:03.972345 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:03.972415 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:03.973314 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.973964 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.975182 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:03.975606 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:03.975709 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:03.977936 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:03.977940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:03.977944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:03.977971 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:03.980174 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.980711 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.980714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.980717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.980841 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:03.981099 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:03.981102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:03.981104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:03.981344 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:03.981539 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:03.981737 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 03:03:03.981896 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 03:03:03.982690 13928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:04.064826 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 13765
I20260807 03:03:04.085078 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 13944
Aug 07 03:03:04 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:03:04.100099 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:04.085975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.5.122.2:41541)
0807 03:03:04.086487 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:04.086494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:04.086519 (+ 25us) server_negotiation.cc:407] Connection header received
0807 03:03:04.086578 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:04.086584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:04.086798 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:04.086943 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:04.087899 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:04.088900 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:04.090077 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:04.090756 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:04.090879 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:04.094732 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:04.094744 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:04.094766 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:04.094800 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:04.097440 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:04.098067 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:04.098073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:04.098076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:04.098198 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:04.098583 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:04.098586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:04.098588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:04.098922 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:04.099216 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:04.099549 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 03:03:04.099809 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":232,"threads_started":1}
I20260807 03:03:04.101289 14113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:04.184114 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 13944
I20260807 03:03:04.206897 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 14124
Aug 07 03:03:04 dist-test-slave-fgb3 krb5kdc[13626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:03:04.223443 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:04.207923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.5.122.3:39475)
0807 03:03:04.208614 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:04.208622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:04.208654 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:03:04.208724 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:04.208731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:04.208943 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:04.209121 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:04.210293 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:04.211387 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:04.212713 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:04.213300 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:04.213428 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:04.217521 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:04.217534 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:04.217547 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:04.217582 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:04.220393 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:04.221116 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:04.221123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:04.221125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:04.221265 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:04.221668 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:04.221672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:04.221674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:04.222039 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:04.222534 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:04.222844 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 03:03:04.223112 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":359,"threads_started":1}
I20260807 03:03:04.224761 14281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:04.305882 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 14124
I20260807 03:03:04.329536 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 13642
I20260807 03:03:04.330803 13722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:04.431105 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 13642
2026-08-07T03:03:04Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3919 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:03:04 dist-test-slave-fgb3 krb5kdc[14332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:03:04 dist-test-slave-fgb3 krb5kdc[14332](info): set up 2 sockets
Aug 07 03:03:04 dist-test-slave-fgb3 krb5kdc[14332](info): commencing operation
krb5kdc: starting...
W20260807 03:03:05.604316 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.009s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:03:05 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab.
Entry for principal oryx/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab.
I20260807 03:03:05.823488 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45857
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab
--principal=oryx/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:45857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:06.130664 14347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:06.130944 14347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:06.131065 14347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:06.139146 14347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:06.139292 14347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:06.139371 14347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:06.139447 14347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:06.139506 14347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:06.149164 14347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:45857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab
--principal=oryx/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:06.150446 14347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:06.151724 14347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:06.162573 14355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:06.162606 14358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:06.162663 14353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:06.163504 14347 server_base.cc:1034] running on GCE node
I20260807 03:03:06.164181 14347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:06.165414 14347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:06.166610 14347 hybrid_clock.cc:648] HybridClock initialized: now 1786071786166554 us; error 41 us; skew 500 ppm
I20260807 03:03:06.166925 14347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:06 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071786, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:06.169893 14347 init.cc:377] Logged in from keytab as oryx/127.5.122.62@KRBTEST.COM (short username oryx)
I20260807 03:03:06.171211 14347 webserver.cc:460] Webserver started at http://127.5.122.62:35243/ using document root <none> and password file <none>
I20260807 03:03:06.171802 14347 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:06.171929 14347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:06.172253 14347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:06.175146 14347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "1a8beeac975047818b5f8a6f109d4882"
format_stamp: "Formatted at 2026-08-07 03:03:06 on dist-test-slave-fgb3"
server_key: "bf2a32cfef8ae4c23bf8ae562542a85c"
server_key_iv: "6bdeb7b781f26c2004001e42b933376f"
server_key_version: "encryptionkey@0"
I20260807 03:03:06.175851 14347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "1a8beeac975047818b5f8a6f109d4882"
format_stamp: "Formatted at 2026-08-07 03:03:06 on dist-test-slave-fgb3"
server_key: "bf2a32cfef8ae4c23bf8ae562542a85c"
server_key_iv: "6bdeb7b781f26c2004001e42b933376f"
server_key_version: "encryptionkey@0"
I20260807 03:03:06.180545 14347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:03:06.183732 14367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:06.184950 14347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 03:03:06.185093 14347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "1a8beeac975047818b5f8a6f109d4882"
format_stamp: "Formatted at 2026-08-07 03:03:06 on dist-test-slave-fgb3"
server_key: "bf2a32cfef8ae4c23bf8ae562542a85c"
server_key_iv: "6bdeb7b781f26c2004001e42b933376f"
server_key_version: "encryptionkey@0"
I20260807 03:03:06.185254 14347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:06.207291 14347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:06.210951 14347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:06.211189 14347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:06.228070 14347 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:45857
I20260807 03:03:06.228112 14441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:45857 every 8 connection(s)
I20260807 03:03:06.229276 14347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:06.230866 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 14347
I20260807 03:03:06.231078 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:03:06.231878 5608 external_mini_cluster.cc:1428] Setting key 950018e5c5a0cee811d2847c0f688276
I20260807 03:03:06.233562 14442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 03:03:06 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:06.246393 14442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: Bootstrap starting.
I20260807 03:03:06.250041 14442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:06.251212 14442 log.cc:824] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:06.254244 14442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: No bootstrap required, opened a new log
I20260807 03:03:06.254494 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:06.233863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.5.122.62:45857)
0807 03:03:06.234611 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:06.234624 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:06.234669 (+ 45us) server_negotiation.cc:407] Connection header received
0807 03:03:06.236596 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:06.236622 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:06.237076 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:06.237481 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:06.238832 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:06.240794 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:06.241918 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:06.242896 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:06.243066 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:06.246941 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:06.246951 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:06.246967 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:06.247015 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:06.250593 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:06.251187 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:06.251195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:06.251207 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:06.251396 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:06.251753 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:06.251759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:06.251772 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:06.252255 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:06.252533 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:06.252888 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 03:03:06.253263 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":295,"threads_started":1}
I20260807 03:03:06.261273 14442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } }
I20260807 03:03:06.261576 14442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:06.261649 14442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8beeac975047818b5f8a6f109d4882, State: Initialized, Role: FOLLOWER
I20260807 03:03:06.262081 14442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } }
I20260807 03:03:06.262275 14442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:06.262405 14442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:06.262529 14442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:06.264448 14442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } }
I20260807 03:03:06.264884 14442 leader_election.cc:302] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a8beeac975047818b5f8a6f109d4882; no voters:
I20260807 03:03:06.265430 14442 leader_election.cc:288] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:03:06.265547 14449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:06.266211 14449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 1 LEADER]: Becoming Leader. State: Replica: 1a8beeac975047818b5f8a6f109d4882, State: Running, Role: LEADER
I20260807 03:03:06.266762 14449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } }
I20260807 03:03:06.267279 14442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:06.269152 14451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8beeac975047818b5f8a6f109d4882. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } }
I20260807 03:03:06.269194 14450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } }
I20260807 03:03:06.269544 14451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:06.269604 14450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:06.274195 14454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:06.278296 14454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:06.285694 14454 catalog_manager.cc:1347] Generated new cluster ID: 7812f86194aa45bd8c28110b55a4fc03
I20260807 03:03:06.285832 14454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:06.292382 14454 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:03:06.293985 14454 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:06.303231 14454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: Generated new TSK 0
I20260807 03:03:06.303869 14454 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab.
Entry for principal oryx/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab.
I20260807 03:03:06.363279 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45857
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab
--principal=oryx/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:06.675726 14479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:06.675993 14479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:06.676121 14479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:06.683269 14479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:06.683391 14479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:06.683553 14479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:06.693315 14479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab
--principal=oryx/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:06.694621 14479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:06.695953 14479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:06.708423 14487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:06.708458 14491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:06.708491 14489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:06.709338 14479 server_base.cc:1034] running on GCE node
I20260807 03:03:06.709934 14479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:06.710844 14479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:06.712033 14479 hybrid_clock.cc:648] HybridClock initialized: now 1786071786711992 us; error 30 us; skew 500 ppm
I20260807 03:03:06.712312 14479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:06 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071786, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:06.715406 14479 init.cc:377] Logged in from keytab as oryx/127.5.122.1@KRBTEST.COM (short username oryx)
I20260807 03:03:06.716734 14479 webserver.cc:460] Webserver started at http://127.5.122.1:42645/ using document root <none> and password file <none>
I20260807 03:03:06.717353 14479 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:06.717471 14479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:06.717746 14479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:06.720544 14479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "3b066723907340d5b2b6cf63912ff335"
format_stamp: "Formatted at 2026-08-07 03:03:06 on dist-test-slave-fgb3"
server_key: "1235570373519c541317baa92ad24566"
server_key_iv: "66de49c483868a1399f01aedec6b15c6"
server_key_version: "encryptionkey@0"
I20260807 03:03:06.721246 14479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "3b066723907340d5b2b6cf63912ff335"
format_stamp: "Formatted at 2026-08-07 03:03:06 on dist-test-slave-fgb3"
server_key: "1235570373519c541317baa92ad24566"
server_key_iv: "66de49c483868a1399f01aedec6b15c6"
server_key_version: "encryptionkey@0"
I20260807 03:03:06.726071 14479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 03:03:06.729278 14505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:06.730543 14479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:06.730687 14479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "3b066723907340d5b2b6cf63912ff335"
format_stamp: "Formatted at 2026-08-07 03:03:06 on dist-test-slave-fgb3"
server_key: "1235570373519c541317baa92ad24566"
server_key_iv: "66de49c483868a1399f01aedec6b15c6"
server_key_version: "encryptionkey@0"
I20260807 03:03:06.730855 14479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:06.749827 14479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:06.754051 14479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:06.754397 14479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:06.755265 14479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:06.756479 14479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:06.756548 14479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:06.756644 14479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:06.756685 14479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:06.788509 14479 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:43645
I20260807 03:03:06.788552 14662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:43645 every 8 connection(s)
I20260807 03:03:06.789785 14479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:06.792758 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 14479
I20260807 03:03:06.792937 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:03:06.793569 5608 external_mini_cluster.cc:1428] Setting key 381f7d29597bb67e393d908300f86f4c
Aug 07 03:03:06 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071786, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:06.813197 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:06.792125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:59713 (local address 127.5.122.62:45857)
0807 03:03:06.792570 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:06.792577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:06.796158 (+ 3581us) server_negotiation.cc:407] Connection header received
0807 03:03:06.798568 (+ 2410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:06.798572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:06.798638 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:06.798728 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:06.800561 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:06.801272 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:06.803379 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:06.803765 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:06.803858 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:06.807898 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:06.807904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:06.807907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:06.807940 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:06.810225 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:06.811052 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:06.811056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:06.811058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:06.811175 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:06.811659 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:06.811661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:06.811663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:06.811917 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:06.812019 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:06.812780 (+ 761us) server_negotiation.cc:299] Negotiation successful
0807 03:03:06.812942 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":208,"threads_started":1}
I20260807 03:03:06.814692 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:06.792647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.1:59713)
0807 03:03:06.795974 (+ 3327us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:06.796042 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:06.798271 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:06.798991 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:06.799002 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:06.799660 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:06.800402 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:06.800425 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:06.801420 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:06.801425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:06.803220 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:06.803235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:06.803988 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:06.803995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:06.804255 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:06.804973 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:06.805006 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:06.807676 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:06.810391 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:06.810400 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:06.810415 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:06.810950 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:06.811273 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:06.811278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:06.811288 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:06.811562 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:06.812037 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:06.812044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:06.812453 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 03:03:06.812729 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 03:03:06.813349 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3033,"thread_start_us":2847,"threads_started":1}
I20260807 03:03:06.816200 14663 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45857
I20260807 03:03:06.816545 14663 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:06.817471 14663 heartbeater.cc:511] Master 127.5.122.62:45857 requested a full tablet report, sending...
I20260807 03:03:06.819653 14388 ts_manager.cc:194] Registered new tserver with Master: 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645)
WARNING: no policy specified for oryx/127.5.122.2@KRBTEST.COM; defaulting to no policy
I20260807 03:03:06.821058 14388 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:59713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab.
Entry for principal oryx/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab.
I20260807 03:03:06.847560 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45857
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab
--principal=oryx/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:07.162909 14674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:07.163192 14674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:07.163341 14674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:07.170564 14674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:07.170678 14674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:07.170827 14674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:07.180619 14674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab
--principal=oryx/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:07.181866 14674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:07.183161 14674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:07.195393 14681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:07.195441 14682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:07.195465 14684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:07.196604 14674 server_base.cc:1034] running on GCE node
I20260807 03:03:07.197158 14674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:07.198112 14674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:07.199303 14674 hybrid_clock.cc:648] HybridClock initialized: now 1786071787199247 us; error 46 us; skew 500 ppm
I20260807 03:03:07.199606 14674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:07 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071787, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:07.202768 14674 init.cc:377] Logged in from keytab as oryx/127.5.122.2@KRBTEST.COM (short username oryx)
I20260807 03:03:07.204073 14674 webserver.cc:460] Webserver started at http://127.5.122.2:34399/ using document root <none> and password file <none>
I20260807 03:03:07.204708 14674 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:07.204806 14674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:07.205044 14674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:07.207940 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "451bb126bdb647a48dd8b37ef76726a3"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "024ab5d52ae431c768344ab5f233b90e"
server_key_iv: "adccab9e5f565c1ac8436a0fd8bb56eb"
server_key_version: "encryptionkey@0"
I20260807 03:03:07.208642 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "451bb126bdb647a48dd8b37ef76726a3"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "024ab5d52ae431c768344ab5f233b90e"
server_key_iv: "adccab9e5f565c1ac8436a0fd8bb56eb"
server_key_version: "encryptionkey@0"
I20260807 03:03:07.213625 14674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 03:03:07.216872 14692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:07.218159 14674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:03:07.218302 14674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "451bb126bdb647a48dd8b37ef76726a3"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "024ab5d52ae431c768344ab5f233b90e"
server_key_iv: "adccab9e5f565c1ac8436a0fd8bb56eb"
server_key_version: "encryptionkey@0"
I20260807 03:03:07.218529 14674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:07.234992 14674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:07.239202 14674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:07.239473 14674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:07.240409 14674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:07.241742 14674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:07.241822 14674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:07.241940 14674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:07.241990 14674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:07.271250 14674 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:43873
I20260807 03:03:07.271289 14834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:43873 every 8 connection(s)
I20260807 03:03:07.272526 14674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:07.279148 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 14674
I20260807 03:03:07.279313 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:03:07.279752 5608 external_mini_cluster.cc:1428] Setting key 28609fff00ce1bed421e609fd8199324
Aug 07 03:03:07 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071787, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:07.298451 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:07.275134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:42963 (local address 127.5.122.62:45857)
0807 03:03:07.275304 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:07.275308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:07.276465 (+ 1157us) server_negotiation.cc:407] Connection header received
0807 03:03:07.278288 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:07.278293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:07.278405 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:07.278494 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:07.280023 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.280960 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.282983 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.283391 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.283488 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:07.292886 (+ 9398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:07.292892 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:07.292895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:07.292930 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:07.295397 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:07.296206 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:07.296210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:07.296212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:07.296345 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:07.296804 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:07.296807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:07.296810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:07.297061 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:07.297165 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:07.297981 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 03:03:07.298169 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:03:07.300015 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:07.275596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:42963)
0807 03:03:07.276317 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:07.276354 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:07.278028 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:07.278735 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:07.278743 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:07.279268 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:07.279847 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:07.279880 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.281112 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.281116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:07.282849 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:07.282866 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.283614 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.283622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:07.283913 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:07.289951 (+ 6038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:07.289979 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:07.292651 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:07.295540 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:07.295549 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:07.295566 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:07.296113 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:07.296443 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:07.296447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:07.296450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:07.296695 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:07.297186 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:07.297193 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:07.297618 (+ 425us) client_negotiation.cc:769] Sending connection context
0807 03:03:07.297923 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 03:03:07.298673 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":283,"threads_started":1}
I20260807 03:03:07.301474 14836 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45857
I20260807 03:03:07.301803 14836 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:07.302789 14836 heartbeater.cc:511] Master 127.5.122.62:45857 requested a full tablet report, sending...
I20260807 03:03:07.304418 14388 ts_manager.cc:194] Registered new tserver with Master: 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2:43873)
WARNING: no policy specified for oryx/127.5.122.3@KRBTEST.COM; defaulting to no policy
I20260807 03:03:07.305171 14388 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:42963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab.
Entry for principal oryx/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab.
I20260807 03:03:07.335048 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45857
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab
--principal=oryx/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:07.647297 14848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:07.647562 14848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:07.647701 14848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:07.654587 14848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:07.654691 14848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:07.654835 14848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:07.663849 14848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab
--principal=oryx/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:45857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:07.664948 14848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:07.666158 14848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:07.678252 14859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:07.678280 14862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:07.678340 14860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:07.679481 14848 server_base.cc:1034] running on GCE node
I20260807 03:03:07.680044 14848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:07.680931 14848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:07.682132 14848 hybrid_clock.cc:648] HybridClock initialized: now 1786071787682079 us; error 39 us; skew 500 ppm
I20260807 03:03:07.682466 14848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:07 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071787, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:07.685425 14848 init.cc:377] Logged in from keytab as oryx/127.5.122.3@KRBTEST.COM (short username oryx)
I20260807 03:03:07.686782 14848 webserver.cc:460] Webserver started at http://127.5.122.3:41073/ using document root <none> and password file <none>
I20260807 03:03:07.687449 14848 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:07.687565 14848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:07.687839 14848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:07.690814 14848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "eefa93d1276e4bbbbe9d49ef727f9bad"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "f740500fa16e3359eec99781213cfbee"
server_key_iv: "2cee83e2143cde5ed4226b7ceef93143"
server_key_version: "encryptionkey@0"
I20260807 03:03:07.691550 14848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "eefa93d1276e4bbbbe9d49ef727f9bad"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "f740500fa16e3359eec99781213cfbee"
server_key_iv: "2cee83e2143cde5ed4226b7ceef93143"
server_key_version: "encryptionkey@0"
I20260807 03:03:07.696540 14848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:03:07.700104 14870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:07.701327 14848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 03:03:07.701483 14848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "eefa93d1276e4bbbbe9d49ef727f9bad"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "f740500fa16e3359eec99781213cfbee"
server_key_iv: "2cee83e2143cde5ed4226b7ceef93143"
server_key_version: "encryptionkey@0"
I20260807 03:03:07.701664 14848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:07.719671 14848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:07.723695 14848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:07.723976 14848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:07.724877 14848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:07.726246 14848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:07.726367 14848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 03:03:07.726506 14848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:07.726576 14848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:07.756665 14848 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:39383
I20260807 03:03:07.756708 15015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:39383 every 8 connection(s)
I20260807 03:03:07.757899 14848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:07.764328 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 14848
I20260807 03:03:07.764500 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:03:07.765066 5608 external_mini_cluster.cc:1428] Setting key dd6a7a258b441973c4e3bdab0b16d1c4
Aug 07 03:03:07 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071787, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:07.776863 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:07.760214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:45977 (local address 127.5.122.62:45857)
0807 03:03:07.760368 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:07.760379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:07.761478 (+ 1099us) server_negotiation.cc:407] Connection header received
0807 03:03:07.763212 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:07.763217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:07.763317 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:07.763401 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:07.765073 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.765987 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.767780 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.768288 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.768381 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:07.771645 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:07.771650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:07.771654 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:07.771687 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:07.774058 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:07.774763 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:07.774767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:07.774769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:07.774890 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:07.775282 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:07.775287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:07.775289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:07.775570 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:07.775701 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:07.776310 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 03:03:07.776524 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 03:03:07.777654 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:07.760654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.3:45977)
0807 03:03:07.761311 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:07.761353 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:07.762963 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:07.763582 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:07.763588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:07.764135 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:07.764892 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:07.764918 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.766123 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.766129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:07.767639 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:07.767654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:07.768483 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:07.768488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:07.768735 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:07.769323 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:07.769350 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:07.771454 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:07.774179 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:07.774187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:07.774201 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:07.774642 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:07.774991 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:07.774996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:07.775000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:07.775170 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:07.775692 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:07.775699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:07.775976 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 03:03:07.776187 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 03:03:07.776642 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":258,"threads_started":1}
I20260807 03:03:07.778960 15016 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45857
I20260807 03:03:07.779228 15016 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:07.780012 15016 heartbeater.cc:511] Master 127.5.122.62:45857 requested a full tablet report, sending...
I20260807 03:03:07.781546 14388 ts_manager.cc:194] Registered new tserver with Master: eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383)
I20260807 03:03:07.782281 14388 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:45977
I20260807 03:03:07.792092 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.122.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.61@KRBTEST.COM" created.
I20260807 03:03:07.824286 14663 heartbeater.cc:503] Master 127.5.122.62:45857 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.61.keytab.
Entry for principal oryx/127.5.122.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.61.keytab.
I20260807 03:03:07.841058 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:45345
--webserver_interface=127.5.122.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.61.keytab
--principal=oryx/127.5.122.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:45857,127.5.122.61:45345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:08.161332 15024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:08.161602 15024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:08.161728 15024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:08.168846 15024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:08.168951 15024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:08.169025 15024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:08.169099 15024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:08.169157 15024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:08.178903 15024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:45857,127.5.122.61:45345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.61.keytab
--principal=oryx/127.5.122.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
W20260807 03:03:08.179234 15024 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.122.62:45857,127.5.122.61:45345'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 03:03:08.180230 15024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:08.181506 15024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:08.192505 15030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:08.192461 15031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:08.192533 15034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:08.193619 15024 server_base.cc:1034] running on GCE node
I20260807 03:03:08.194437 15024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:08.195755 15024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:08.196962 15024 hybrid_clock.cc:648] HybridClock initialized: now 1786071788196891 us; error 43 us; skew 500 ppm
I20260807 03:03:08.197268 15024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:08 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:08.200467 15024 init.cc:377] Logged in from keytab as oryx/127.5.122.61@KRBTEST.COM (short username oryx)
I20260807 03:03:08.201862 15024 webserver.cc:460] Webserver started at http://127.5.122.61:42601/ using document root <none> and password file <none>
I20260807 03:03:08.202592 15024 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:08.202696 15024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:08.203004 15024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:08.206002 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/instance:
uuid: "f862fb4523e3488eada0c8a770e48fe1"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "c701ed76eb62c62e9bc020992698644e"
server_key_iv: "a0b18ac7d76b5022537b0cfebd674d8a"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.206861 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal/instance:
uuid: "f862fb4523e3488eada0c8a770e48fe1"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "c701ed76eb62c62e9bc020992698644e"
server_key_iv: "a0b18ac7d76b5022537b0cfebd674d8a"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.212178 15024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:03:08.215719 15048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:08.217140 15024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:03:08.217306 15024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
uuid: "f862fb4523e3488eada0c8a770e48fe1"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "c701ed76eb62c62e9bc020992698644e"
server_key_iv: "a0b18ac7d76b5022537b0cfebd674d8a"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.217510 15024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:08.240533 15024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:08.244782 15024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:08.245044 15024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:08.262591 15024 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.61:45345
I20260807 03:03:08.262614 15117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.61:45345 every 8 connection(s)
I20260807 03:03:08.263926 15024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
I20260807 03:03:08.267673 15118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:08.270442 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 15024
I20260807 03:03:08.276177 15118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:03:08 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.61@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
Aug 07 03:03:08 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.61@KRBTEST.COM
I20260807 03:03:08.297026 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.279471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.5.122.62:45857)
0807 03:03:08.279986 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.279992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.280581 (+ 589us) server_negotiation.cc:407] Connection header received
0807 03:03:08.281031 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.281036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.281111 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.281234 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:08.282903 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.284005 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.285914 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.286449 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.286572 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.290458 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:08.290466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:08.290470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:08.290508 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:08.293710 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.294544 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.294549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.294553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.294782 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.295184 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.295189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.295193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.295625 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:08.295766 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.296447 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.296670 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":279,"threads_started":1}
I20260807 03:03:08.298009 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.272757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.5.122.61:45345)
0807 03:03:08.278552 (+ 5795us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.278566 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.278719 (+ 153us) server_negotiation.cc:407] Connection header received
0807 03:03:08.280814 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.280836 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.281086 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.281282 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:08.283139 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.284603 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.286175 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.286936 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.287036 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.291315 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:08.291327 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:08.291342 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:08.291389 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:08.294646 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.295221 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.295228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.295231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.295367 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.295703 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.295707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.295710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.296078 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:08.296324 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.296675 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.296970 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":5559,"thread_start_us":5006,"threads_started":1}
I20260807 03:03:08.298044 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.279435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:45756)
0807 03:03:08.280424 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:08.280464 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:08.280805 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:08.281320 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:08.281326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:08.282188 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:08.282736 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.282761 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.284130 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.284133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.285770 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.285785 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.286655 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.286660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.286817 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.287536 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:08.287572 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:08.290253 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:08.293853 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.293862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.293871 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.294431 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.294889 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.294894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.294898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.295068 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.295733 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:08.295739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:08.296048 (+ 309us) client_negotiation.cc:769] Sending connection context
0807 03:03:08.296315 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 03:03:08.296832 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":554,"threads_started":1}
I20260807 03:03:08.300020 15118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:03:08 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.61@KRBTEST.COM for oryx/127.5.122.61@KRBTEST.COM
I20260807 03:03:08.307600 14836 heartbeater.cc:503] Master 127.5.122.62:45857 was elected leader, sending a full tablet report...
I20260807 03:03:08.311872 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.300660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.0.0.1:60986)
0807 03:03:08.300830 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:08.300847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:08.300948 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:08.301204 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:08.301207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:08.301520 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:08.301649 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.301657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.302604 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.302607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.303450 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.303461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.304056 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.304061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.304206 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.304660 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:08.304675 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:08.306837 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:08.309675 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.309680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.309684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.310023 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.310371 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.310376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.310380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.310505 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.310979 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:08.310984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:08.311103 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 03:03:08.311254 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 03:03:08.311521 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 03:03:08.311969 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.300804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.5.122.61:45345)
0807 03:03:08.300980 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.300985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.301000 (+ 15us) server_negotiation.cc:407] Connection header received
0807 03:03:08.301038 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.301043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.301095 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.301178 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:08.301759 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.302488 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.303563 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.303946 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.304034 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.307007 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:08.307012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:08.307015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:08.307042 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:08.309560 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.310122 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.310126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.310128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.310239 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.310599 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.310603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.310606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.310867 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:08.310999 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.311336 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.311651 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":45}
I20260807 03:03:08.316428 15118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Bootstrap starting.
I20260807 03:03:08.319414 15118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:08.320351 15118 log.cc:824] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:08.323050 15118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: No bootstrap required, opened a new log
I20260807 03:03:08.329698 15118 raft_consensus.cc:348] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } }
I20260807 03:03:08.329988 15118 raft_consensus.cc:374] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:08.330056 15118 raft_consensus.cc:729] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f862fb4523e3488eada0c8a770e48fe1, State: Initialized, Role: FOLLOWER
I20260807 03:03:08.330551 15118 consensus_queue.cc:260] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } }
I20260807 03:03:08.331291 15130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } } }
I20260807 03:03:08.331586 15130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:03:08.332152 15118 sys_catalog.cc:564] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 03:03:08.339896 15149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 03:03:08.340011 15149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:03:08.343438 15024 master_runner.cc:428] Detected that this master f862fb4523e3488eada0c8a770e48fe1 is joining an existing cluster
I20260807 03:03:08.343521 15024 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.122.61:45345
I20260807 03:03:08.362715 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.347235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.5.122.61:45345)
0807 03:03:08.347403 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.347409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.348543 (+ 1134us) server_negotiation.cc:407] Connection header received
0807 03:03:08.348690 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.348695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.348758 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.348861 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:08.350108 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.351110 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.353275 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.353944 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.354032 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.356597 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:08.356602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:08.356606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:08.356635 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:08.359950 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.360643 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.360648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.360650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.360788 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.361227 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.361231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.361233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.361556 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:08.361672 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.362148 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.362386 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 03:03:08.363725 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.347238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.0.0.1:60998)
0807 03:03:08.348435 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:08.348452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:08.348581 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:08.348895 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:08.348899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:08.349631 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:08.349966 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.349978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.351279 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.351284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.353125 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.353140 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.354300 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.354305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.354501 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.355067 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:08.355094 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:08.356452 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:08.360072 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.360076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.360079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.360540 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.360921 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.360925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.360928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.361133 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.361743 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:08.361747 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:08.361906 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 03:03:08.363077 (+ 1171us) client_negotiation.cc:240] Negotiation successful
0807 03:03:08.363351 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1071,"mutex_wait_us":4,"spinlock_wait_cycles":57984,"thread_start_us":885,"threads_started":1}
I20260807 03:03:08.364018 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.347602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45762 (local address 127.5.122.62:45857)
0807 03:03:08.347772 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.347777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.348053 (+ 276us) server_negotiation.cc:407] Connection header received
0807 03:03:08.348328 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.348334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.348437 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.348493 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:08.349815 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.350838 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.352950 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.353597 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.353695 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.356725 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:08.356732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:08.356737 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:08.356776 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:08.360764 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.361500 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.361506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.361509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.361657 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.362111 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.362116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.362120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.362514 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:08.362631 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.363454 (+ 823us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.363680 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:03:08.365036 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.347193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:45762)
0807 03:03:08.347942 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:08.347963 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:08.348166 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:08.348690 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:08.348695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:08.349388 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:08.349679 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.349687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.350987 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.350992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.352771 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.352787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.353984 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.353991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.354175 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.355057 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:08.355080 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:08.356513 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:08.360917 (+ 4404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.360923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.360926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.361356 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.361808 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.361813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.361816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.361976 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.362730 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:08.362735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:08.363121 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 03:03:08.364263 (+ 1142us) client_negotiation.cc:240] Negotiation successful
0807 03:03:08.364574 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":45,"thread_start_us":581,"threads_started":1}
Aug 07 03:03:08 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071786, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.62@KRBTEST.COM for oryx/127.5.122.61@KRBTEST.COM
I20260807 03:03:08.384066 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.370799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.5.122.61:45345)
0807 03:03:08.370971 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.370977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.371258 (+ 281us) server_negotiation.cc:407] Connection header received
0807 03:03:08.371569 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.371572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.371652 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.371728 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:08.372924 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.373715 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.375042 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.375525 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.375635 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.378770 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:08.378775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:08.378778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:08.378807 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:08.381423 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.382099 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.382103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.382105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.382232 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.382648 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.382651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.382654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.382920 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:08.383055 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.383527 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.383724 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:03:08.384122 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.370521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.0.0.1:32776)
0807 03:03:08.371134 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:08.371177 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:08.371389 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:08.371793 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:08.371800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:08.372509 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:08.372788 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.372799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.373829 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.373834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.374908 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.374926 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.375671 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.375677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.375869 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.376503 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:08.376526 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:08.378601 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:08.381542 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.381559 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.381567 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.381982 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.382371 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.382376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.382379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.382533 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.383043 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:08.383051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:08.383211 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 03:03:08.383366 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 03:03:08.383680 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":258,"threads_started":1}
WARNING: no policy specified for oryx/127.5.122.60@KRBTEST.COM; defaulting to no policy
I20260807 03:03:08.386425 14388 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.122.61:45345: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1"
member_type: NON_VOTER
last_known_addr {
host: "127.5.122.61"
port: 45345
}
attrs {
promote: true
}
}
dest_uuid: "1a8beeac975047818b5f8a6f109d4882"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.60@KRBTEST.COM" created.
I20260807 03:03:08.388049 14388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } }
I20260807 03:03:08.401752 15083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:08.404479 15083 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Refusing update from remote peer 1a8beeac975047818b5f8a6f109d4882: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 03:03:08.400837 15165 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f862fb4523e3488eada0c8a770e48fe1 (127.5.122.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } }
I20260807 03:03:08.408036 15163 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:08.408257 15165 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.122.61:45345
I20260807 03:03:08.409228 15024 master.cc:537] Master@127.5.122.61:45345 shutting down...
I20260807 03:03:08.410852 15167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } }
I20260807 03:03:08.411237 15167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
W20260807 03:03:08.412295 14371 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 03:03:08.418639 14371 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 -> Peer f862fb4523e3488eada0c8a770e48fe1 (127.5.122.61:45345): Couldn't send request to peer f862fb4523e3488eada0c8a770e48fe1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 03:03:08.423009 15024 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:08.423444 15024 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:08.423626 15024 tablet_replica.cc:330] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.60.keytab.
Entry for principal oryx/127.5.122.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.60.keytab.
I20260807 03:03:08.433732 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:36569
--webserver_interface=127.5.122.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.60.keytab
--principal=oryx/127.5.122.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:03:08.441087 15024 master.cc:552] Master@127.5.122.61:45345 shutdown complete.
I20260807 03:03:08.441231 15024 master_runner.cc:305] Clearing existing system tablet
I20260807 03:03:08.441716 15024 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:08.441835 15024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:08.444950 15024 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:03:08.446695 15175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:08.447026 15024 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:08.447288 15024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 03:03:08.447367 15024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
uuid: "f862fb4523e3488eada0c8a770e48fe1"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "c701ed76eb62c62e9bc020992698644e"
server_key_iv: "a0b18ac7d76b5022537b0cfebd674d8a"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.452807 15024 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:08.459653 15024 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 03:03:08.459825 15024 log.cc:1196] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 03:03:08.460269 15024 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Deleting consensus metadata
I20260807 03:03:08.460528 15024 master_runner.cc:315] Copying system tablet from 127.5.122.62:45857
I20260807 03:03:08.463338 15024 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: tablet copy: Beginning tablet copy session from remote peer at address 127.5.122.62:45857
I20260807 03:03:08.480584 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.464514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.5.122.62:45857)
0807 03:03:08.464696 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.464702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.466834 (+ 2132us) server_negotiation.cc:407] Connection header received
0807 03:03:08.467070 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.467077 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.467159 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.467271 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:08.468288 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.469456 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.471343 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.471929 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.472051 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.474369 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:08.474377 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:08.474382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:08.474423 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:08.477718 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.478504 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.478510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.478513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.478663 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:08.479058 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:08.479063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:08.479066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:08.479411 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:08.479563 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.480021 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.480234 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:03:08.480644 15182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.464424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:45778)
0807 03:03:08.466697 (+ 2273us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:08.466727 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:08.466887 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:08.467316 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:08.467322 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:08.467843 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:08.468123 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.468136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.469611 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.469617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.471108 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.471123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.472126 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.472132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.472312 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.473018 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:08.473041 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:08.474138 (+ 1097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:08.477849 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.477854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.477859 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.478396 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.478767 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:08.478772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:08.478776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:08.478949 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:08.479552 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:08.479557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:08.479727 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 03:03:08.479868 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 03:03:08.480219 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2131,"thread_start_us":2020,"threads_started":1}
I20260807 03:03:08.481781 14417 tablet_copy_service.cc:140] P 1a8beeac975047818b5f8a6f109d4882: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f862fb4523e3488eada0c8a770e48fe1 ({username='oryx', principal='oryx/127.5.122.61@KRBTEST.COM'} at 127.0.0.1:45778)
I20260807 03:03:08.482137 14417 tablet_copy_service.cc:161] P 1a8beeac975047818b5f8a6f109d4882: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f862fb4523e3488eada0c8a770e48fe1 at {username='oryx', principal='oryx/127.5.122.61@KRBTEST.COM'} at 127.0.0.1:45778: session id = f862fb4523e3488eada0c8a770e48fe1-00000000000000000000000000000000
I20260807 03:03:08.485414 14417 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 03:03:08.488095 15024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 03:03:08.494884 15024 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: tablet copy: Starting download of 0 data blocks...
I20260807 03:03:08.495268 15024 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: tablet copy: Starting download of 1 WAL segments...
I20260807 03:03:08.499159 15024 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 03:03:08.503710 14417 tablet_copy_service.cc:342] P 1a8beeac975047818b5f8a6f109d4882: Request end of tablet copy session f862fb4523e3488eada0c8a770e48fe1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.122.61@KRBTEST.COM'} at 127.0.0.1:45778
I20260807 03:03:08.503991 14417 tablet_copy_service.cc:433] P 1a8beeac975047818b5f8a6f109d4882: ending tablet copy session f862fb4523e3488eada0c8a770e48fe1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f862fb4523e3488eada0c8a770e48fe1
I20260807 03:03:08.516467 15024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:08.518923 15183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:08.522933 15185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:08.530051 15024 server_base.cc:1034] running on GCE node
W20260807 03:03:08.530614 15189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:08.531215 15024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:08.531878 15024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:08.533093 15024 hybrid_clock.cc:648] HybridClock initialized: now 1786071788533034 us; error 36 us; skew 500 ppm
I20260807 03:03:08.533309 15024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:08 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:08.535552 15024 init.cc:377] Logged in from keytab as oryx/127.5.122.61@KRBTEST.COM (short username oryx)
I20260807 03:03:08.539537 15024 webserver.cc:460] Webserver started at http://127.5.122.61:41707/ using document root <none> and password file <none>
I20260807 03:03:08.539999 15024 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:08.540153 15024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:08.543737 15024 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260807 03:03:08.546100 15199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:08.546576 15024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 03:03:08.546718 15024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
uuid: "f862fb4523e3488eada0c8a770e48fe1"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "c701ed76eb62c62e9bc020992698644e"
server_key_iv: "a0b18ac7d76b5022537b0cfebd674d8a"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.546881 15024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:08.563535 15024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:08.568385 15024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:08.628034 15024 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.61:45345
I20260807 03:03:08.628094 15270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.61:45345 every 8 connection(s)
I20260807 03:03:08.629364 15024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
I20260807 03:03:08.635845 15271 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:03:08.640617 15271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Bootstrap starting.
I20260807 03:03:08.662777 15271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:03:08.663450 15271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: Bootstrap complete.
I20260807 03:03:08.664776 15271 raft_consensus.cc:348] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } }
I20260807 03:03:08.665096 15271 raft_consensus.cc:729] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f862fb4523e3488eada0c8a770e48fe1, State: Initialized, Role: LEARNER
I20260807 03:03:08.665433 15271 consensus_queue.cc:260] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } }
I20260807 03:03:08.666868 15271 sys_catalog.cc:564] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:08.666689 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } }
I20260807 03:03:08.667266 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: LEARNER
I20260807 03:03:08.679230 15293 catalog_manager.cc:1259] Loaded cluster ID: 7812f86194aa45bd8c28110b55a4fc03
I20260807 03:03:08.679364 15293 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: loading cluster ID for follower catalog manager: success
I20260807 03:03:08.683125 15293 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: acquiring CA information for follower catalog manager: success
I20260807 03:03:08.686075 15293 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 03:03:08.784564 15016 heartbeater.cc:503] Master 127.5.122.62:45857 was elected leader, sending a full tablet report...
I20260807 03:03:08.856683 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.849901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.5.122.61:45345)
0807 03:03:08.850501 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:08.850505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:08.850526 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:03:08.850584 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:08.850587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:08.850652 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:08.850750 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:08.851310 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.852387 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.854796 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.855749 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.855845 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.856144 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:08.856340 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 03:03:08.856438 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":317,"threads_started":1}
I20260807 03:03:08.856739 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:08.849657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.0.0.1:32780)
0807 03:03:08.849875 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:08.849897 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:08.850037 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:08.850852 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:08.850857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:08.850912 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:08.851145 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.851160 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.852535 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.852541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.854657 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:08.854673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:08.855926 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:08.855932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:08.856162 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:08.856222 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 03:03:08.856325 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 03:03:08.856461 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 03:03:08.858408 15236 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 03:03:08.859089 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8beeac975047818b5f8a6f109d4882. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } }
I20260807 03:03:08.859339 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: LEARNER
W20260807 03:03:08.876192 15173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:08.876474 15173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:08.876642 15173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:08.884439 15173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:08.884541 15173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:08.884624 15173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:08.884704 15173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:08.884783 15173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:08.895120 15173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.60.keytab
--principal=oryx/127.5.122.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:36569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:08.896456 15173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:08.897796 15173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:08.910565 15302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:08.910843 15305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:08.910938 15303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:08.911737 15173 server_base.cc:1034] running on GCE node
I20260807 03:03:08.912627 15173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:08.913929 15173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:08.915149 15173 hybrid_clock.cc:648] HybridClock initialized: now 1786071788915081 us; error 40 us; skew 500 ppm
I20260807 03:03:08.915499 15173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:08 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:08.918577 15173 init.cc:377] Logged in from keytab as oryx/127.5.122.60@KRBTEST.COM (short username oryx)
I20260807 03:03:08.920045 15173 webserver.cc:460] Webserver started at http://127.5.122.60:43557/ using document root <none> and password file <none>
I20260807 03:03:08.920662 15173 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:08.920787 15173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:08.921113 15173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:08.924405 15173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/instance:
uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "7d535dde89044f75c9e1fbba2d3db2f8"
server_key_iv: "c68a4b18e78dc3a1b002fb2b049aa464"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.925186 15173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal/instance:
uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "7d535dde89044f75c9e1fbba2d3db2f8"
server_key_iv: "c68a4b18e78dc3a1b002fb2b049aa464"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.930361 15173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 03:03:08.933687 15314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:08.934973 15173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 03:03:08.935148 15173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "7d535dde89044f75c9e1fbba2d3db2f8"
server_key_iv: "c68a4b18e78dc3a1b002fb2b049aa464"
server_key_version: "encryptionkey@0"
I20260807 03:03:08.935308 15173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:08.967042 15173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:08.971938 15173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:08.972266 15173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:08.995739 15173 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.60:36569
I20260807 03:03:08.995795 15386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.60:36569 every 8 connection(s)
I20260807 03:03:08.997187 15173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
I20260807 03:03:08.999475 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 15173
I20260807 03:03:09.001726 15387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:09.010703 15387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.60@KRBTEST.COM
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.60@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:09.024282 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.001851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.5.122.60:36569)
0807 03:03:09.003021 (+ 1170us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.003036 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.003099 (+ 63us) server_negotiation.cc:407] Connection header received
0807 03:03:09.005166 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.005186 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.005547 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.005894 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.007640 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.009126 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.010563 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.011331 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.011493 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.016062 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.016070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.016078 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.016112 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.019746 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.020426 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.020433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.020439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.020656 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.021105 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.021111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.021115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.021785 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.022207 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.022545 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.022986 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":897,"thread_start_us":541,"threads_started":1}
I20260807 03:03:09.029817 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.013313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.5.122.62:45857)
0807 03:03:09.013904 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.013911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.014477 (+ 566us) server_negotiation.cc:407] Connection header received
0807 03:03:09.014717 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.014721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.014790 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.014953 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.016127 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.016903 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.019252 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.019669 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.019759 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.023623 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.023628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.023632 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.023663 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.027031 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.027890 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.027894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.027896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.028028 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.028467 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.028471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.028473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.028781 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.028890 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.029326 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.029541 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":333,"threads_started":1}
I20260807 03:03:09.029850 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.013232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:45784)
0807 03:03:09.014344 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.014384 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.014576 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.015022 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.015028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.015687 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.015983 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.015994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.017069 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.017075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.019123 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.019139 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.019877 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.019884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.020019 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.020677 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.020698 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.023410 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.027155 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.027166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.027172 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.027775 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.028113 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.028117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.028119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.028351 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.028887 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.028893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.029061 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.029224 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.029490 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":741,"threads_started":1}
I20260807 03:03:09.031239 15387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.60@KRBTEST.COM for oryx/127.5.122.61@KRBTEST.COM
I20260807 03:03:09.044369 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.032009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.0.0.1:32788)
0807 03:03:09.032206 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.032226 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.032323 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.032570 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.032573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.032955 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.033175 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.033185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.034421 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.034425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.035625 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.035638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.036431 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.036436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.036637 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.037109 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.037123 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.039246 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.042516 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.042521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.042524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.042883 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.043207 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.043210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.043213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.043322 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.043739 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.043742 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.043855 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.043962 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.044152 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 03:03:09.044523 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.032074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.5.122.61:45345)
0807 03:03:09.032232 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.032236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.032285 (+ 49us) server_negotiation.cc:407] Connection header received
0807 03:03:09.032408 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.032412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.032472 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.032546 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.033367 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.034234 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.035752 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.036260 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.036360 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.039436 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.039440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.039443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.039476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.042396 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.042968 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.042971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.042973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.043132 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.043394 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.043396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.043398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.043661 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.043772 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.044026 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.044209 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:03:09.045511 15387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071788, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.60@KRBTEST.COM for oryx/127.5.122.60@KRBTEST.COM
I20260807 03:03:09.058228 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.046104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.0.0.1:55612)
0807 03:03:09.046239 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.046253 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.046353 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.046627 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.046631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.047054 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.047223 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.047231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.048357 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.048361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.049321 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.049333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.050036 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.050042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.050251 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.050792 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.050807 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.053005 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.055779 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.055784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.055789 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.056221 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.056599 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.056604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.056608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.056781 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.057255 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.057260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.057440 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.057552 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.057777 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 03:03:09.058250 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.046249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55612 (local address 127.5.122.60:36569)
0807 03:03:09.046425 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.046430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.046445 (+ 15us) server_negotiation.cc:407] Connection header received
0807 03:03:09.046479 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.046482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.046526 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.046600 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.047341 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.048220 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.049449 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.049855 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.049959 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.053154 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.053158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.053161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.053187 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.055670 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.056332 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.056336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.056338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.056494 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.056877 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.056883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.056886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.057149 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.057272 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.057644 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.057933 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":74}
I20260807 03:03:09.063422 15387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Bootstrap starting.
I20260807 03:03:09.066826 15387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:09.068096 15387 log.cc:824] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:09.070901 15387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: No bootstrap required, opened a new log
I20260807 03:03:09.078788 15387 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } }
I20260807 03:03:09.079106 15387 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:09.079216 15387 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ab14c14ed04dfda8dffd59f0b5d10d, State: Initialized, Role: FOLLOWER
I20260807 03:03:09.079738 15387 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } }
I20260807 03:03:09.080700 15396 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } } }
I20260807 03:03:09.081166 15396 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:03:09.081615 15387 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: configured and running, proceeding with master startup.
W20260807 03:03:09.091508 15412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 03:03:09.091630 15412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:03:09.095558 15173 master_runner.cc:428] Detected that this master c7ab14c14ed04dfda8dffd59f0b5d10d is joining an existing cluster
I20260807 03:03:09.095695 15173 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.122.60:36569
I20260807 03:03:09.119956 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.100866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.5.122.62:45857)
0807 03:03:09.101034 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.101039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.102101 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 03:03:09.102384 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.102389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.102476 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.102553 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.104511 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.105637 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.108014 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.108741 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.108839 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.112260 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.112280 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.112285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.112324 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.116342 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.117098 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.117103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.117107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.117287 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.117904 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.117909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.117914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.118336 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.118459 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.119240 (+ 781us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.119548 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 03:03:09.122144 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.100773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:45800)
0807 03:03:09.101996 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.102016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.102205 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.102754 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.102759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.103644 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.104321 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.104336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.105815 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.105820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.107831 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.107850 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.109162 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.109171 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.109414 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.110595 (+ 1181us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.110617 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.112031 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.116481 (+ 4450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.116486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.116489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.116971 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.117434 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.117438 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.117442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.117772 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.118562 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.118567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.118903 (+ 336us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.121068 (+ 2165us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.121395 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1071,"spinlock_wait_cycles":34688,"thread_start_us":881,"threads_started":1}
I20260807 03:03:09.126466 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.100709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.5.122.61:45345)
0807 03:03:09.100924 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.100929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.108177 (+ 7248us) server_negotiation.cc:407] Connection header received
0807 03:03:09.108717 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.108725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.108819 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.108885 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.111576 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.112567 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.116198 (+ 3631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.116856 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.116952 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.119539 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.119545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.119550 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.119588 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.123124 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.124005 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.124010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.124013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.124216 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.124683 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.124688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.124691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.125041 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.125163 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.125844 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.126057 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:03:09.126654 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.101164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.0.0.1:32802)
0807 03:03:09.108066 (+ 6902us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.108087 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.108559 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.109102 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.109107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.110717 (+ 1610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.111397 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.111411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.112754 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.112758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.116021 (+ 3263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.116039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.117236 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.117243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.117409 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.118088 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.118109 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.119342 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.123269 (+ 3927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.123273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.123277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.123859 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.124360 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.124364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.124367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.124546 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.125241 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.125245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.125539 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.125691 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.125985 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6718,"mutex_wait_us":83,"thread_start_us":7019,"threads_started":1}
I20260807 03:03:09.130730 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.102473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.5.122.60:36569)
0807 03:03:09.102640 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.102646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.110513 (+ 7867us) server_negotiation.cc:407] Connection header received
0807 03:03:09.110701 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.110705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.110770 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.110839 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.112738 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.113889 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.116109 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.117009 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.117119 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.120464 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.120475 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.120480 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.120516 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.125111 (+ 4595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.128389 (+ 3278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.128394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.128398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.128535 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.128964 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.128968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.128972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.129371 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.129530 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.130103 (+ 573us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.130307 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 03:03:09.131626 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.102145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.0.0.1:55626)
0807 03:03:09.110425 (+ 8280us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.110443 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.110592 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.111052 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.111056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.112138 (+ 1082us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.112599 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.112611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.114074 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.114079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.115956 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.115971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.117503 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.117514 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.117857 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.118497 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.118538 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.119980 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.125251 (+ 5271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.125255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.125259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.128131 (+ 2872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.128665 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.128670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.128673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.128821 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.129597 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.129601 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.129766 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.130728 (+ 962us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.131031 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8102,"mutex_wait_us":36,"thread_start_us":7700,"threads_started":1}
I20260807 03:03:09.131966 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 14479
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071786, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.62@KRBTEST.COM for oryx/127.5.122.60@KRBTEST.COM
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.1@KRBTEST.COM
I20260807 03:03:09.149550 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.128676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55642 (local address 127.5.122.60:36569)
0807 03:03:09.131464 (+ 2788us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.131469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.131485 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:03:09.131573 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.131578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.132709 (+ 1131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.134238 (+ 1529us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.135368 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.136601 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.138441 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.139025 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.139130 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.143018 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.143025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.143029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.143063 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.146763 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.147438 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.147443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.147447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.147604 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.148004 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.148009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.148013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.148377 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.148515 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.148915 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.149127 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2231,"thread_start_us":1982,"threads_started":1}
I20260807 03:03:09.149551 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.128370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.0.0.1:55642)
0807 03:03:09.128775 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.128794 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.128898 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.134347 (+ 5449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.134354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.134920 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.135184 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.135199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.136764 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.136770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.138303 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.138342 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.139206 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.139213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.139370 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.140450 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.140474 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.142812 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.146920 (+ 4108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.146926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.146930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.147298 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.147753 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.147757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.147759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.147901 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.148511 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.148516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.148663 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.148780 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.149068 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260807 03:03:09.151252 14388 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.122.60:36569: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d"
member_type: NON_VOTER
last_known_addr {
host: "127.5.122.60"
port: 36569
}
attrs {
promote: true
}
}
dest_uuid: "1a8beeac975047818b5f8a6f109d4882"
cas_config_opid_index: 5
I20260807 03:03:09.152385 14388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } }
I20260807 03:03:09.154392 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.133387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.5.122.1:43645)
0807 03:03:09.134331 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.134341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.134371 (+ 30us) server_negotiation.cc:407] Connection header received
0807 03:03:09.134495 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.134506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.134738 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.134899 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.136312 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.137318 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.138609 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.139668 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.139797 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.145768 (+ 5971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.145784 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.145814 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.145855 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.150495 (+ 4640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.151331 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.151341 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.151344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.151470 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.151890 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.151894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.151896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.152248 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.153507 (+ 1259us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.153686 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.153973 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":506,"threads_started":1}
I20260807 03:03:09.154577 15167 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c7ab14c14ed04dfda8dffd59f0b5d10d (127.5.122.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.157220 15167 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.122.60:36569
I20260807 03:03:09.157891 15173 master.cc:537] Master@127.5.122.60:36569 shutting down...
I20260807 03:03:09.158432 15236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 LEARNER]: Refusing update from remote peer 1a8beeac975047818b5f8a6f109d4882: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 03:03:09.159072 15354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:09.159207 14656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:09.163431 15434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 03:03:09.164656 15435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.165012 15435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:09.169320 15236 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c7ab14c14ed04dfda8dffd59f0b5d10d (127.5.122.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.174906 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: NON_VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: true } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.175196 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: LEARNER
I20260807 03:03:09.177469 15434 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: attempting to promote NON_VOTER f862fb4523e3488eada0c8a770e48fe1 to VOTER
I20260807 03:03:09.178695 15434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } }
I20260807 03:03:09.179610 15354 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Refusing update from remote peer 1a8beeac975047818b5f8a6f109d4882: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 03:03:09.182655 15236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 LEARNER]: Refusing update from remote peer 1a8beeac975047818b5f8a6f109d4882: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 03:03:09.183379 15167 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 03:03:09.186440 15167 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f862fb4523e3488eada0c8a770e48fe1 (127.5.122.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.187983 15234 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f862fb4523e3488eada0c8a770e48fe1 (127.5.122.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.190936 15167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.191651 15167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:09.191359 15434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.191922 15434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:09.197638 15173 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:09.198103 15173 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:09.198997 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.199237 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:03:09.199775 15173 tablet_replica.cc:330] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: stopping tablet replica
W20260807 03:03:09.199852 14369 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 -> Peer c7ab14c14ed04dfda8dffd59f0b5d10d (127.5.122.60:36569): Couldn't send request to peer c7ab14c14ed04dfda8dffd59f0b5d10d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 03:03:09.205827 15173 master.cc:552] Master@127.5.122.60:36569 shutdown complete.
I20260807 03:03:09.205892 15173 master_runner.cc:305] Clearing existing system tablet
I20260807 03:03:09.206194 15173 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:09.206283 15173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:09.208567 15173 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 03:03:09.209867 15443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:09.210186 15173 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:09.210489 15173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 03:03:09.210595 15173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "7d535dde89044f75c9e1fbba2d3db2f8"
server_key_iv: "c68a4b18e78dc3a1b002fb2b049aa464"
server_key_version: "encryptionkey@0"
I20260807 03:03:09.213835 15173 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:09.218477 15173 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 03:03:09.218632 15173 log.cc:1196] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 03:03:09.219013 15173 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Deleting consensus metadata
I20260807 03:03:09.219219 15173 master_runner.cc:315] Copying system tablet from 127.5.122.62:45857
I20260807 03:03:09.223058 15173 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: tablet copy: Beginning tablet copy session from remote peer at address 127.5.122.62:45857
I20260807 03:03:09.253351 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.223775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45810 (local address 127.5.122.62:45857)
0807 03:03:09.224049 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.224056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.238842 (+ 14786us) server_negotiation.cc:407] Connection header received
0807 03:03:09.239068 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.239073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.239168 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.239258 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.240283 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.241466 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.243365 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.244119 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.244208 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.246759 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.246765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.246770 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.246808 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.250474 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.251284 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.251290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.251294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.251450 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.251874 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.251879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.251883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.252216 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.252349 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.252774 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.252977 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260807 03:03:09.253693 15449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.223706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:45810)
0807 03:03:09.238725 (+ 15019us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.238750 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.238922 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.239287 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.239291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.239872 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.240113 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.240127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.241655 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.241661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.243202 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.243217 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.244487 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.244493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.244689 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.245401 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.245425 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.246540 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.250636 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.250640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.250644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.251151 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.251565 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.251570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.251574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.251758 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.252335 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.252340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.252507 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.252623 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.252978 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14871,"thread_start_us":15329,"threads_started":1}
I20260807 03:03:09.254267 14417 tablet_copy_service.cc:140] P 1a8beeac975047818b5f8a6f109d4882: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c7ab14c14ed04dfda8dffd59f0b5d10d ({username='oryx', principal='oryx/127.5.122.60@KRBTEST.COM'} at 127.0.0.1:45810)
I20260807 03:03:09.254463 14417 tablet_copy_service.cc:161] P 1a8beeac975047818b5f8a6f109d4882: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c7ab14c14ed04dfda8dffd59f0b5d10d at {username='oryx', principal='oryx/127.5.122.60@KRBTEST.COM'} at 127.0.0.1:45810: session id = c7ab14c14ed04dfda8dffd59f0b5d10d-00000000000000000000000000000000
I20260807 03:03:09.256916 14417 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 03:03:09.258512 15173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 03:03:09.263875 15173 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: tablet copy: Starting download of 0 data blocks...
I20260807 03:03:09.264164 15173 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: tablet copy: Starting download of 1 WAL segments...
I20260807 03:03:09.266278 15173 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 03:03:09.269618 14417 tablet_copy_service.cc:342] P 1a8beeac975047818b5f8a6f109d4882: Request end of tablet copy session c7ab14c14ed04dfda8dffd59f0b5d10d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.122.60@KRBTEST.COM'} at 127.0.0.1:45810
I20260807 03:03:09.269727 14417 tablet_copy_service.cc:433] P 1a8beeac975047818b5f8a6f109d4882: ending tablet copy session c7ab14c14ed04dfda8dffd59f0b5d10d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c7ab14c14ed04dfda8dffd59f0b5d10d
I20260807 03:03:09.279117 15173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:09.281376 15450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:09.287187 15451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:09.292330 15173 server_base.cc:1034] running on GCE node
W20260807 03:03:09.292721 15453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:09.293087 15173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:09.293718 15173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:09.294896 15173 hybrid_clock.cc:648] HybridClock initialized: now 1786071789294851 us; error 42 us; skew 500 ppm
I20260807 03:03:09.295099 15173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071789, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:09.296890 15173 init.cc:377] Logged in from keytab as oryx/127.5.122.60@KRBTEST.COM (short username oryx)
I20260807 03:03:09.299180 15173 webserver.cc:460] Webserver started at http://127.5.122.60:38911/ using document root <none> and password file <none>
I20260807 03:03:09.299461 15173 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:09.299557 15173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:09.301963 15173 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 03:03:09.303844 15461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:09.304286 15173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 03:03:09.304504 15173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d"
format_stamp: "Formatted at 2026-08-07 03:03:08 on dist-test-slave-fgb3"
server_key: "7d535dde89044f75c9e1fbba2d3db2f8"
server_key_iv: "c68a4b18e78dc3a1b002fb2b049aa464"
server_key_version: "encryptionkey@0"
I20260807 03:03:09.304809 15173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:09.307478 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 14479
I20260807 03:03:09.331517 15173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:09.336020 15173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:09.339236 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:43645
--local_ip_for_outbound_sockets=127.5.122.1
--tserver_master_addrs=127.5.122.62:45857
--webserver_port=42645
--webserver_interface=127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab
--principal=oryx/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:03:09.382141 15173 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.60:36569
I20260807 03:03:09.382208 15532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.60:36569 every 8 connection(s)
I20260807 03:03:09.383222 15173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
I20260807 03:03:09.388404 15534 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:03:09.392221 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Bootstrap starting.
I20260807 03:03:09.413697 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:03:09.414239 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Bootstrap complete.
I20260807 03:03:09.415390 15534 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } }
I20260807 03:03:09.415688 15534 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7ab14c14ed04dfda8dffd59f0b5d10d, State: Initialized, Role: LEARNER
I20260807 03:03:09.415961 15534 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } }
I20260807 03:03:09.417193 15538 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.417506 15534 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:09.417630 15538 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: This master's current role is: LEARNER
I20260807 03:03:09.431164 15555 catalog_manager.cc:1259] Loaded cluster ID: 7812f86194aa45bd8c28110b55a4fc03
I20260807 03:03:09.431368 15555 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: loading cluster ID for follower catalog manager: success
I20260807 03:03:09.434624 15555 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: acquiring CA information for follower catalog manager: success
I20260807 03:03:09.436934 15555 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 03:03:09.632584 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.624508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.0.0.1:55644)
0807 03:03:09.624802 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.624823 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.624959 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.625862 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.625867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.625905 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:09.626103 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.626118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.628033 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.628038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.629911 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.629930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.631787 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.631794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.631984 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.632024 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.632129 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.632274 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260807 03:03:09.632663 15556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.624701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.5.122.60:36569)
0807 03:03:09.625481 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.625487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.625510 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:03:09.625582 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.625587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.625672 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.625813 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:09.626271 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.627865 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.630099 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.631608 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.631728 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.631986 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.632169 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.632304 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":471,"threads_started":1}
I20260807 03:03:09.633971 15498 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 03:03:09.634617 15538 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8beeac975047818b5f8a6f109d4882. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:09.634915 15538 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: This master's current role is: LEARNER
W20260807 03:03:09.706260 15487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:09.706606 15487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:09.706773 15487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:09.715003 15487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:09.715137 15487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:09.715324 15487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:09.726521 15487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab
--principal=oryx/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:43645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=42645
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:09.727878 15487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:09.729296 15487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:09.743425 15562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:09.743481 15566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:09.744424 15564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:09.747635 15487 server_base.cc:1034] running on GCE node
I20260807 03:03:09.748286 15487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:09.749264 15487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:09.750470 15487 hybrid_clock.cc:648] HybridClock initialized: now 1786071789750401 us; error 49 us; skew 500 ppm
I20260807 03:03:09.750797 15487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071789, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:09.754096 15487 init.cc:377] Logged in from keytab as oryx/127.5.122.1@KRBTEST.COM (short username oryx)
I20260807 03:03:09.755672 15487 webserver.cc:460] Webserver started at http://127.5.122.1:42645/ using document root <none> and password file <none>
I20260807 03:03:09.756417 15487 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:09.756568 15487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:09.762984 15487 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 03:03:09.766003 15576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:09.767356 15487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:09.767537 15487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "3b066723907340d5b2b6cf63912ff335"
format_stamp: "Formatted at 2026-08-07 03:03:06 on dist-test-slave-fgb3"
server_key: "1235570373519c541317baa92ad24566"
server_key_iv: "66de49c483868a1399f01aedec6b15c6"
server_key_version: "encryptionkey@0"
I20260807 03:03:09.768589 15487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:09.789489 15487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:09.793949 15487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:09.794246 15487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:09.795562 15487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:09.797024 15487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:09.797109 15487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:09.797202 15487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:09.797286 15487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:09.842108 15487 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:43645
I20260807 03:03:09.842199 15736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:43645 every 8 connection(s)
I20260807 03:03:09.843654 15487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:09.854182 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 15487
I20260807 03:03:09.854605 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 14674
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071789, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071789, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.60@KRBTEST.COM
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071789, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.61@KRBTEST.COM
I20260807 03:03:09.881165 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.846566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:36683 (local address 127.5.122.62:45857)
0807 03:03:09.861921 (+ 15355us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.861929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.861952 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:03:09.862028 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.862035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.862124 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.862243 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.864267 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.865016 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.866912 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.867572 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.867696 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.872869 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.872876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.872880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.872920 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.876798 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.877688 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.877693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.877696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.877858 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.878445 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.878449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.878452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.879205 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.879336 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.880529 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.880743 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15194,"thread_start_us":15883,"threads_started":1}
Aug 07 03:03:09 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.2@KRBTEST.COM
I20260807 03:03:09.883718 15751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.847269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.1:36683)
0807 03:03:09.848764 (+ 1495us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.848836 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.852638 (+ 3802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.862555 (+ 9917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.862568 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.863427 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.864076 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.864095 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.865177 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.865180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.866681 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.866697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.868054 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.868064 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.868374 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.869181 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.869214 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.872579 (+ 3365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.876951 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.876959 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.876975 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.877537 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.878012 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.878016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.878020 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.878277 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.879406 (+ 1129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.879413 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.879842 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.880158 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.882204 (+ 2046us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"thread_start_us":774,"threads_started":1}
I20260807 03:03:09.886161 15739 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45857
I20260807 03:03:09.886430 15556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.849857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:37857 (local address 127.5.122.60:36569)
0807 03:03:09.850052 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.850060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.868081 (+ 18021us) server_negotiation.cc:407] Connection header received
0807 03:03:09.868497 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.868503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.868588 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.868666 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.871587 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.872641 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.874626 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.875265 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.875371 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.879497 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.879504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.879509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.879548 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.882855 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.883868 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.883874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.883878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.884046 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.884571 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.884576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.884579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.884942 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.885058 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.885880 (+ 822us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.886113 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 03:03:09.886607 15739 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:09.887178 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.849343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.5.122.1:37857)
0807 03:03:09.867972 (+ 18629us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.867995 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.868235 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.869237 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.869242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.869999 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.870599 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.870620 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.872808 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.872813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.874455 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.874469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.875690 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.875697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.875851 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.876381 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.876402 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.879194 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.882998 (+ 3804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.883003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.883006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.883726 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.884266 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.884270 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.884285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.884447 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.885353 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.885358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.885506 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.886651 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.886885 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18456,"thread_start_us":17252,"threads_started":1}
I20260807 03:03:09.887792 15739 heartbeater.cc:511] Master 127.5.122.62:45857 requested a full tablet report, sending...
I20260807 03:03:09.888296 15738 heartbeater.cc:347] Connected to a master server at 127.5.122.60:36569
I20260807 03:03:09.888430 15738 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:09.888995 15738 heartbeater.cc:511] Master 127.5.122.60:36569 requested a full tablet report, sending...
I20260807 03:03:09.889650 14388 ts_manager.cc:194] Re-registered known tserver with Master: 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645)
I20260807 03:03:09.890796 14388 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:36683
I20260807 03:03:09.891001 15484 ts_manager.cc:194] Registered new tserver with Master: 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645)
I20260807 03:03:09.893319 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.855602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36954 (local address 127.5.122.2:43873)
0807 03:03:09.863120 (+ 7518us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.863128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.868397 (+ 5269us) server_negotiation.cc:407] Connection header received
0807 03:03:09.868861 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.868868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.869096 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.869445 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.872389 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.874496 (+ 2107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.876602 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.877530 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.877643 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.885021 (+ 7378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.885040 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.885057 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.885104 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.890117 (+ 5013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.890985 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.890995 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.890998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.891167 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.891690 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.891697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.891700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.892095 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.892405 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.892688 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.892951 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7341,"thread_start_us":8606,"threads_started":1}
I20260807 03:03:09.894204 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.847269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.5.122.1:53825)
0807 03:03:09.867009 (+ 19740us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:09.867029 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:09.867182 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:09.869482 (+ 2300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:09.869487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:09.871425 (+ 1938us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:09.871959 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.871975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.874079 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.874085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.876140 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:09.876158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.877962 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.877969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:09.878221 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.878835 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:09.878857 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:09.881978 (+ 3121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:09.890925 (+ 8947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.890930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.890934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.891476 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.891959 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:09.891965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:09.891969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:09.892252 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:09.893030 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:09.893035 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:09.893194 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 03:03:09.893366 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 03:03:09.893676 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19515,"mutex_wait_us":36,"spinlock_wait_cycles":47488,"thread_start_us":20068,"threads_started":1}
I20260807 03:03:09.894213 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:09.846638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:53825 (local address 127.5.122.61:45345)
0807 03:03:09.868930 (+ 22292us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:09.868937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:09.868958 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:03:09.869027 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:09.869033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:09.869141 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:09.869268 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:09.872174 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.873896 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.876331 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:09.877582 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:09.877702 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:09.882251 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:09.882259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:09.882264 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:09.882308 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:09.890747 (+ 8439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.891629 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.891634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.891638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.891806 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:09.892408 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:09.892413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:09.892417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:09.892806 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:09.892957 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:09.893522 (+ 565us) server_negotiation.cc:299] Negotiation successful
0807 03:03:09.893723 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22166,"thread_start_us":22364,"threads_started":1}
I20260807 03:03:09.894785 14830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:09.895402 15740 heartbeater.cc:347] Connected to a master server at 127.5.122.61:45345
I20260807 03:03:09.895550 15740 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:09.896101 15740 heartbeater.cc:511] Master 127.5.122.61:45345 requested a full tablet report, sending...
I20260807 03:03:09.897698 15221 ts_manager.cc:194] Registered new tserver with Master: 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645)
I20260807 03:03:09.979050 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 14674
I20260807 03:03:10.002205 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:43873
--local_ip_for_outbound_sockets=127.5.122.2
--tserver_master_addrs=127.5.122.62:45857
--webserver_port=34399
--webserver_interface=127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab
--principal=oryx/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:03:10.205572 15435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: NON_VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: true } } }
I20260807 03:03:10.205960 15435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:10.207468 15759 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: attempting to promote NON_VOTER c7ab14c14ed04dfda8dffd59f0b5d10d to VOTER
I20260807 03:03:10.208033 15759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } }
I20260807 03:03:10.210059 15234 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Refusing update from remote peer 1a8beeac975047818b5f8a6f109d4882: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 03:03:10.210106 15498 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 LEARNER]: Refusing update from remote peer 1a8beeac975047818b5f8a6f109d4882: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 03:03:10.210722 15435 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 03:03:10.211144 15760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 03:03:10.214897 15234 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c7ab14c14ed04dfda8dffd59f0b5d10d (127.5.122.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.217963 15498 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c7ab14c14ed04dfda8dffd59f0b5d10d (127.5.122.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.218623 15765 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.214208 15435 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c7ab14c14ed04dfda8dffd59f0b5d10d (127.5.122.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.219043 15765 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:03:10.222709 15759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.222790 15435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.222991 15759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:10.223039 15435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:10.223193 15759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.223433 15759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:10.224673 15767 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a8beeac975047818b5f8a6f109d4882" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:10.224963 15767 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: This master's current role is: FOLLOWER
W20260807 03:03:10.329092 15758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:10.329366 15758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:10.329492 15758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:10.336474 15758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:10.336598 15758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:10.336750 15758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:10.346167 15758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab
--principal=oryx/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:43873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=34399
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:10.347507 15758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:10.348814 15758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:10.360993 15774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:10.360963 15775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:10.361024 15778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:10.362079 15758 server_base.cc:1034] running on GCE node
I20260807 03:03:10.362665 15758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:10.363519 15758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:10.364717 15758 hybrid_clock.cc:648] HybridClock initialized: now 1786071790364658 us; error 53 us; skew 500 ppm
I20260807 03:03:10.365012 15758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:10 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:10.368434 15758 init.cc:377] Logged in from keytab as oryx/127.5.122.2@KRBTEST.COM (short username oryx)
I20260807 03:03:10.369721 15758 webserver.cc:460] Webserver started at http://127.5.122.2:34399/ using document root <none> and password file <none>
I20260807 03:03:10.370397 15758 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:10.370512 15758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:10.375726 15758 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:03:10.378386 15786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:10.379653 15758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:10.379801 15758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "451bb126bdb647a48dd8b37ef76726a3"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "024ab5d52ae431c768344ab5f233b90e"
server_key_iv: "adccab9e5f565c1ac8436a0fd8bb56eb"
server_key_version: "encryptionkey@0"
I20260807 03:03:10.380755 15758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:10.405985 15758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:10.410146 15758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:10.410476 15758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:10.411505 15758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:10.412873 15758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:10.412973 15758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:10.413081 15758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:10.413153 15758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:10.443743 15758 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:43873
I20260807 03:03:10.443828 15946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:43873 every 8 connection(s)
I20260807 03:03:10.445315 15758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:10.454846 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 15758
I20260807 03:03:10.455184 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 14848
Aug 07 03:03:10 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
Aug 07 03:03:10 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.61@KRBTEST.COM
Aug 07 03:03:10 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.60@KRBTEST.COM
Aug 07 03:03:10 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.3@KRBTEST.COM
I20260807 03:03:10.490546 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:10.448342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:60539 (local address 127.5.122.62:45857)
0807 03:03:10.460753 (+ 12411us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:10.460760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:10.463058 (+ 2298us) server_negotiation.cc:407] Connection header received
0807 03:03:10.465154 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:10.465160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:10.465246 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:10.465362 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:10.468203 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.469100 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.472616 (+ 3516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.473172 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.473309 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:10.480424 (+ 7115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:10.480431 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:10.480436 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:10.480501 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:10.484708 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.487266 (+ 2558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.487271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.487275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.487456 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.488279 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.488309 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.488313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.488656 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:10.488841 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:10.489894 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0807 03:03:10.490114 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12264,"thread_start_us":12403,"threads_started":1}
I20260807 03:03:10.491163 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:10.448213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:52081 (local address 127.5.122.61:45345)
0807 03:03:10.455524 (+ 7311us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:10.455531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:10.465661 (+ 10130us) server_negotiation.cc:407] Connection header received
0807 03:03:10.465761 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:10.465768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:10.465867 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:10.466014 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:10.469353 (+ 3339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.471030 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.474001 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.474911 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.475049 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:10.481208 (+ 6159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:10.481215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:10.481219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:10.481275 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:10.486567 (+ 5292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.487506 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.487512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.487516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.487762 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.488314 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.488320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.488324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.488715 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:10.488884 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:10.490176 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0807 03:03:10.490432 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7173,"thread_start_us":6976,"threads_started":1}
I20260807 03:03:10.491930 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:10.448869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.5.122.2:52081)
0807 03:03:10.462267 (+ 13398us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:10.462330 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:10.464717 (+ 2387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:10.466066 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:10.466071 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:10.466768 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:10.467982 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:10.468010 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.471258 (+ 3248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.471263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:10.473788 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:10.473805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.475517 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.475526 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:10.475832 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:10.476610 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:10.476644 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:10.480978 (+ 4334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:10.486766 (+ 5788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:10.486772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:10.486776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:10.487348 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:10.487909 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:10.487914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:10.487918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:10.488178 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:10.488962 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:10.488966 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:10.489436 (+ 470us) client_negotiation.cc:769] Sending connection context
0807 03:03:10.489822 (+ 386us) client_negotiation.cc:240] Negotiation successful
0807 03:03:10.490458 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6679,"mutex_wait_us":164,"spinlock_wait_cycles":87168,"thread_start_us":13577,"threads_started":1}
I20260807 03:03:10.492717 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:10.456760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:52033 (local address 127.5.122.60:36569)
0807 03:03:10.470788 (+ 14028us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:10.470795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:10.470815 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:03:10.470879 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:10.470884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:10.470964 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:10.471141 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:10.473026 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.474873 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.477371 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.477866 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.477968 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:10.483191 (+ 5223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:10.483198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:10.483202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:10.483242 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:10.486827 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.487755 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.487761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.487764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.487964 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.489217 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.489222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.489225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.489575 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:10.489704 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:10.491951 (+ 2247us) server_negotiation.cc:299] Negotiation successful
0807 03:03:10.492167 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13900,"thread_start_us":13788,"threads_started":1}
I20260807 03:03:10.492023 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:10.460687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.5.122.2:52033)
0807 03:03:10.469669 (+ 8982us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:10.469690 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:10.470089 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:10.471395 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:10.471401 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:10.472371 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:10.472862 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:10.472878 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.475000 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.475005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:10.477241 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:10.477253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.478119 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.478127 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:10.478305 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:10.478921 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:10.478937 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:10.482989 (+ 4052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:10.486958 (+ 3969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:10.486964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:10.486968 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:10.487616 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:10.488098 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:10.488103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:10.488107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:10.489082 (+ 975us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:10.490819 (+ 1737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:10.490824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:10.490958 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 03:03:10.491123 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 03:03:10.491390 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8807,"mutex_wait_us":27,"thread_start_us":8633,"threads_started":1}
I20260807 03:03:10.492772 15957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:10.448869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:60539)
0807 03:03:10.462878 (+ 14009us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:10.462896 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:10.464733 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:10.465623 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:10.465632 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:10.466760 (+ 1128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:10.467982 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:10.468010 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.469773 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.469778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:10.472479 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:10.472503 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.474058 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.474064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:10.474244 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:10.475151 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:10.475189 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:10.480164 (+ 4975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:10.485832 (+ 5668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:10.485842 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:10.485857 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:10.486646 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:10.487762 (+ 1116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:10.487767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:10.487771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:10.488147 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:10.488805 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:10.488813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:10.489436 (+ 623us) client_negotiation.cc:769] Sending connection context
0807 03:03:10.489822 (+ 386us) client_negotiation.cc:240] Negotiation successful
0807 03:03:10.492424 (+ 2602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13851,"mutex_wait_us":89,"spinlock_wait_cycles":5504,"thread_start_us":6530,"threads_started":1}
I20260807 03:03:10.493848 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:10.456346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.5.122.3:39383)
0807 03:03:10.463590 (+ 7244us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:10.463599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:10.468849 (+ 5250us) server_negotiation.cc:407] Connection header received
0807 03:03:10.469010 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:10.469018 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:10.469277 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:10.469456 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:10.473302 (+ 3846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.474984 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.477149 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:10.478179 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:10.478293 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:10.486230 (+ 7937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:10.486245 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:10.486260 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:10.486335 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:10.490257 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.491123 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.491132 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.491135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.491293 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:10.491758 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:10.491763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:10.491766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:10.492159 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:10.492451 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:10.493099 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 03:03:10.493318 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7060,"thread_start_us":8677,"threads_started":1}
I20260807 03:03:10.494451 15952 heartbeater.cc:347] Connected to a master server at 127.5.122.61:45345
I20260807 03:03:10.494521 15949 heartbeater.cc:347] Connected to a master server at 127.5.122.60:36569
I20260807 03:03:10.494839 15949 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:10.494851 15952 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:10.495272 15950 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45857
I20260807 03:03:10.495438 15950 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:10.495910 15952 heartbeater.cc:511] Master 127.5.122.61:45345 requested a full tablet report, sending...
I20260807 03:03:10.495955 15950 heartbeater.cc:511] Master 127.5.122.62:45857 requested a full tablet report, sending...
I20260807 03:03:10.496429 15011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:10.496945 15949 heartbeater.cc:511] Master 127.5.122.60:36569 requested a full tablet report, sending...
I20260807 03:03:10.497833 14388 ts_manager.cc:194] Re-registered known tserver with Master: 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2:43873)
I20260807 03:03:10.498229 15484 ts_manager.cc:194] Registered new tserver with Master: 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2:43873)
I20260807 03:03:10.498238 15221 ts_manager.cc:194] Registered new tserver with Master: 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2:43873)
I20260807 03:03:10.498894 14388 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:60539
I20260807 03:03:10.582646 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 14848
I20260807 03:03:10.603153 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:39383
--local_ip_for_outbound_sockets=127.5.122.3
--tserver_master_addrs=127.5.122.62:45857
--webserver_port=41073
--webserver_interface=127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab
--principal=oryx/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:03:10.894474 15739 heartbeater.cc:503] Master 127.5.122.62:45857 was elected leader, sending a full tablet report...
W20260807 03:03:10.936223 15965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:10.936486 15965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:10.936618 15965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:10.943691 15965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:10.943796 15965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:10.943959 15965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:10.953112 15965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab
--principal=oryx/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:39383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=41073
--tserver_master_addrs=127.5.122.62:45857,127.5.122.61:45345,127.5.122.60:36569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:10.954288 15965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:10.955504 15965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:10.967473 15973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:10.967873 15972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:10.968043 15975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:10.968376 15965 server_base.cc:1034] running on GCE node
I20260807 03:03:10.968896 15965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:10.969743 15965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:10.970942 15965 hybrid_clock.cc:648] HybridClock initialized: now 1786071790970887 us; error 53 us; skew 500 ppm
I20260807 03:03:10.971263 15965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:10 dist-test-slave-fgb3 krb5kdc[14332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:10.974258 15965 init.cc:377] Logged in from keytab as oryx/127.5.122.3@KRBTEST.COM (short username oryx)
I20260807 03:03:10.975627 15965 webserver.cc:460] Webserver started at http://127.5.122.3:41073/ using document root <none> and password file <none>
I20260807 03:03:10.976256 15965 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:10.976397 15965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:10.981333 15965 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 03:03:10.983830 15985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:10.985064 15965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 03:03:10.985206 15965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "eefa93d1276e4bbbbe9d49ef727f9bad"
format_stamp: "Formatted at 2026-08-07 03:03:07 on dist-test-slave-fgb3"
server_key: "f740500fa16e3359eec99781213cfbee"
server_key_iv: "2cee83e2143cde5ed4226b7ceef93143"
server_key_version: "encryptionkey@0"
I20260807 03:03:10.986160 15965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:10.998668 15965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:11.001971 15965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:11.002177 15965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:11.003033 15965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:11.004230 15965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:11.004320 15965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:11.004395 15965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:11.004447 15965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:11.034649 15965 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:39383
I20260807 03:03:11.034709 16145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:39383 every 8 connection(s)
I20260807 03:03:11.036007 15965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:11.045645 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 15965
Aug 07 03:03:11 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.61@KRBTEST.COM
Aug 07 03:03:11 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.60@KRBTEST.COM
Aug 07 03:03:11 dist-test-slave-fgb3 krb5kdc[14332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071790, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:11.066996 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.038839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53163 (local address 127.5.122.61:45345)
0807 03:03:11.040418 (+ 1579us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.040425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.042352 (+ 1927us) server_negotiation.cc:407] Connection header received
0807 03:03:11.043507 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.043513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.043611 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.043706 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.045383 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.046550 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.049254 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.049897 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.050005 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.055794 (+ 5789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.055802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.055806 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.055846 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.061385 (+ 5539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.063311 (+ 1926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.063317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.063320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.063661 (+ 341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.065096 (+ 1435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.065102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.065105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.065580 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.065725 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.066334 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.066558 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1450,"thread_start_us":1327,"threads_started":1}
I20260807 03:03:11.067184 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.038747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53033 (local address 127.5.122.60:36569)
0807 03:03:11.040200 (+ 1453us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.040208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.041148 (+ 940us) server_negotiation.cc:407] Connection header received
0807 03:03:11.043413 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.043419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.043508 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.043600 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.045354 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.046551 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.049261 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.050081 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.050183 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.055779 (+ 5596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.055786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.055793 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.055836 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.061719 (+ 5883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.063070 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.063077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.063081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.063592 (+ 511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.064354 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.064360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.064364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.064961 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.065081 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.066249 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.066515 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1302,"thread_start_us":1192,"threads_started":1}
I20260807 03:03:11.068219 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.039422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.5.122.3:53163)
0807 03:03:11.042202 (+ 2780us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.042222 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.043172 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.043891 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.043896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.044760 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:11.045192 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.045206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.046739 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.046744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.049079 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.049095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.050309 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.050333 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.050616 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.051306 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:11.051342 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:11.055533 (+ 4191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:11.062184 (+ 6651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:11.062189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:11.062192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:11.063161 (+ 969us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:11.064649 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:11.064653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:11.064656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:11.064913 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:11.065821 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:11.065825 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:11.066008 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.066191 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.066728 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2615,"mutex_wait_us":112,"spinlock_wait_cycles":14080,"thread_start_us":15288,"threads_started":1}
I20260807 03:03:11.069047 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.039422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.5.122.3:53033)
0807 03:03:11.040986 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.041029 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.043155 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.043831 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.043839 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.044516 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:11.045130 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.045169 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.046781 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.046786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.049084 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.049101 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.050505 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.050512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.050709 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.051433 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:11.051457 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:11.055158 (+ 3701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:11.061912 (+ 6754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:11.061921 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:11.061940 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:11.062916 (+ 976us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:11.063779 (+ 863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:11.063784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:11.063788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:11.064203 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:11.065242 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:11.065251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:11.065720 (+ 469us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.067450 (+ 1730us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.067987 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1287,"mutex_wait_us":95,"spinlock_wait_cycles":12544,"thread_start_us":1042,"threads_started":1}
I20260807 03:03:11.069908 16148 heartbeater.cc:347] Connected to a master server at 127.5.122.61:45345
I20260807 03:03:11.070248 16148 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:11.070271 16146 heartbeater.cc:347] Connected to a master server at 127.5.122.60:36569
I20260807 03:03:11.070535 16146 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:11.071376 16146 heartbeater.cc:511] Master 127.5.122.60:36569 requested a full tablet report, sending...
I20260807 03:03:11.071398 16148 heartbeater.cc:511] Master 127.5.122.61:45345 requested a full tablet report, sending...
I20260807 03:03:11.073513 15484 ts_manager.cc:194] Registered new tserver with Master: eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383)
I20260807 03:03:11.075062 15221 ts_manager.cc:194] Registered new tserver with Master: eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383)
I20260807 03:03:11.076715 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.041554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.3:55713)
0807 03:03:11.055200 (+ 13646us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.055219 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.055376 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.055889 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.055895 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.056455 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:11.056753 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.056765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.058256 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.058262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.060581 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.060597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.062490 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.062501 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.062891 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.063648 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:11.063670 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:11.069548 (+ 5878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:11.073476 (+ 3928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:11.073481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:11.073485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:11.073988 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:11.074976 (+ 988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:11.074981 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:11.074984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:11.075233 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:11.075784 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:11.075789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:11.075925 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.076033 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.076325 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13494,"thread_start_us":303,"threads_started":1}
I20260807 03:03:11.076720 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.041669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:55713 (local address 127.5.122.62:45857)
0807 03:03:11.054698 (+ 13029us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.054705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.055314 (+ 609us) server_negotiation.cc:407] Connection header received
0807 03:03:11.055498 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.055504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.055574 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.055685 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.056958 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.058085 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.060761 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.061912 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.062040 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.069760 (+ 7720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.069767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.069772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.069814 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.073295 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.074128 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.074135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.074139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.074857 (+ 718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.075378 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.075383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.075385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.075661 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.075816 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.076162 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.076382 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12901,"thread_start_us":12803,"threads_started":1}
I20260807 03:03:11.077847 16147 heartbeater.cc:347] Connected to a master server at 127.5.122.62:45857
I20260807 03:03:11.077970 16147 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:11.078389 16147 heartbeater.cc:511] Master 127.5.122.62:45857 requested a full tablet report, sending...
I20260807 03:03:11.079308 14388 ts_manager.cc:194] Re-registered known tserver with Master: eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383)
I20260807 03:03:11.080370 14388 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:55713
I20260807 03:03:11.097466 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.080416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.5.122.62:45857)
0807 03:03:11.081677 (+ 1261us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.081682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.081721 (+ 39us) server_negotiation.cc:407] Connection header received
0807 03:03:11.081774 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.081779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.081928 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.082052 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.084649 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.085822 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.087456 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.088094 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.088215 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.091509 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.091516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.091521 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.091561 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.094587 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.095401 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.095407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.095411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.095559 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.095974 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.095980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.095983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.096281 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.096526 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.096890 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.097118 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1130}
I20260807 03:03:11.098841 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.081443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.5.122.60:36569)
0807 03:03:11.081610 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.081615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.084031 (+ 2416us) server_negotiation.cc:407] Connection header received
0807 03:03:11.084184 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.084190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.084316 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.084413 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.086691 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.087503 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.089322 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.089825 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.089932 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.092865 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.092873 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.092877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.092918 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.095875 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.096685 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.096690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.096693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.096853 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.097387 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.097392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.097396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.097714 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.097985 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.098298 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.098543 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:03:11.100164 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.079970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.5.122.61:45345)
0807 03:03:11.080150 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.080155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.081765 (+ 1610us) server_negotiation.cc:407] Connection header received
0807 03:03:11.082132 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.082138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.082211 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.082294 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.085855 (+ 3561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.086929 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.088935 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.089569 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.089727 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.094079 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.094086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.094090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.094131 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.097215 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.097938 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.097944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.097947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.098119 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.098580 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.098586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.098589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.098875 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.099077 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.099638 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.099860 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 03:03:11.102185 14388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:03:11.109138 14388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:11.137676 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.127349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39383 (local address 127.0.0.1:47778)
0807 03:03:11.127930 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.127951 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.128192 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.130439 (+ 2247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.130442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.130463 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:11.130763 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.130773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.132810 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.132815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.134449 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.134467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.136941 (+ 2474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.136947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.137110 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.137142 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.137225 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.137350 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":284,"threads_started":1}
I20260807 03:03:11.137863 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.127739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.5.122.3:39383)
0807 03:03:11.129734 (+ 1995us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.129743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.129778 (+ 35us) server_negotiation.cc:407] Connection header received
0807 03:03:11.129891 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.129901 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.130164 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.130471 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:11.130937 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.132691 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.134606 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.136780 (+ 2174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.136906 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.137115 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.137411 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.137571 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1824,"thread_start_us":1725,"threads_started":1}
I20260807 03:03:11.141423 16054 tablet_service.cc:1467] Processing CreateTablet for tablet 3932cc7a53974395ae6fa610d4ce390a (DEFAULT_TABLE table=test-table [id=51501770ec5a44dab2fb1a290680e19c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:11.142267 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.128030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43645 (local address 127.0.0.1:33638)
0807 03:03:11.134750 (+ 6720us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.134768 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.135198 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.136101 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.136105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.136128 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:11.136551 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.136565 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.138200 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.138205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.140223 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.140238 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.141400 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.141407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.141609 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.141641 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.141764 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.141888 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6606,"mutex_wait_us":24,"thread_start_us":6483,"threads_started":1}
I20260807 03:03:11.142292 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.128412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33638 (local address 127.5.122.1:43645)
0807 03:03:11.129392 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.129402 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.135074 (+ 5672us) server_negotiation.cc:407] Connection header received
0807 03:03:11.135417 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.135426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.135669 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.135831 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:11.136752 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.138012 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.140343 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.141202 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.141330 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.141532 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.141831 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.142001 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":463,"threads_started":1}
I20260807 03:03:11.142982 16054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3932cc7a53974395ae6fa610d4ce390a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:11.144766 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.134826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:43873 (local address 127.0.0.1:36964)
0807 03:03:11.135369 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.135386 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.135705 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.136498 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.136504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.136533 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:11.138086 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.138100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.139896 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.139901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.142051 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.142067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.143978 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.143985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.144160 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.144192 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.144287 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.144406 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":1,"thread_start_us":275,"threads_started":1}
I20260807 03:03:11.144783 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.135085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36964 (local address 127.5.122.2:43873)
0807 03:03:11.135636 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.135643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.135670 (+ 27us) server_negotiation.cc:407] Connection header received
0807 03:03:11.135862 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.135869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.136090 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.136474 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:11.138265 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.139751 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.142231 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.143757 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.143886 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.144078 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.144341 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.144501 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":246,"threads_started":1}
I20260807 03:03:11.145686 15648 tablet_service.cc:1467] Processing CreateTablet for tablet 3932cc7a53974395ae6fa610d4ce390a (DEFAULT_TABLE table=test-table [id=51501770ec5a44dab2fb1a290680e19c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:11.147089 15648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3932cc7a53974395ae6fa610d4ce390a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:11.148561 15851 tablet_service.cc:1467] Processing CreateTablet for tablet 3932cc7a53974395ae6fa610d4ce390a (DEFAULT_TABLE table=test-table [id=51501770ec5a44dab2fb1a290680e19c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:11.150223 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3932cc7a53974395ae6fa610d4ce390a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:11.155961 16195 tablet_bootstrap.cc:492] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Bootstrap starting.
I20260807 03:03:11.159562 16195 tablet_bootstrap.cc:654] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:11.160594 16195 log.cc:824] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:11.163151 16197 tablet_bootstrap.cc:492] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Bootstrap starting.
I20260807 03:03:11.164122 16195 tablet_bootstrap.cc:492] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: No bootstrap required, opened a new log
I20260807 03:03:11.164402 16195 ts_tablet_manager.cc:1397] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 03:03:11.165483 16199 tablet_bootstrap.cc:492] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Bootstrap starting.
I20260807 03:03:11.166687 16197 tablet_bootstrap.cc:654] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:11.168540 16197 log.cc:824] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:11.168606 16199 tablet_bootstrap.cc:654] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:11.169700 16199 log.cc:824] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:11.171728 16197 tablet_bootstrap.cc:492] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: No bootstrap required, opened a new log
I20260807 03:03:11.172016 16197 ts_tablet_manager.cc:1397] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 03:03:11.174212 16195 raft_consensus.cc:348] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.174787 16195 raft_consensus.cc:374] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:11.174906 16195 raft_consensus.cc:729] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eefa93d1276e4bbbbe9d49ef727f9bad, State: Initialized, Role: FOLLOWER
I20260807 03:03:11.175583 16195 consensus_queue.cc:260] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.176542 16199 tablet_bootstrap.cc:492] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: No bootstrap required, opened a new log
I20260807 03:03:11.176730 16147 heartbeater.cc:503] Master 127.5.122.62:45857 was elected leader, sending a full tablet report...
I20260807 03:03:11.176791 16199 ts_tablet_manager.cc:1397] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 03:03:11.177358 16195 ts_tablet_manager.cc:1428] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260807 03:03:11.183240 16197 raft_consensus.cc:348] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.183650 16197 raft_consensus.cc:374] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:11.183755 16197 raft_consensus.cc:729] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b066723907340d5b2b6cf63912ff335, State: Initialized, Role: FOLLOWER
I20260807 03:03:11.184343 16197 consensus_queue.cc:260] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.185950 16197 ts_tablet_manager.cc:1428] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260807 03:03:11.186137 16199 raft_consensus.cc:348] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.186554 16199 raft_consensus.cc:374] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:11.186654 16199 raft_consensus.cc:729] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451bb126bdb647a48dd8b37ef76726a3, State: Initialized, Role: FOLLOWER
I20260807 03:03:11.187192 16199 consensus_queue.cc:260] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.189298 16199 ts_tablet_manager.cc:1428] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Time spent starting tablet: real 0.012s user 0.013s sys 0.001s
I20260807 03:03:11.189558 15950 heartbeater.cc:503] Master 127.5.122.62:45857 was elected leader, sending a full tablet report...
W20260807 03:03:11.198539 15953 tablet.cc:2369] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:03:11.278251 16208 raft_consensus.cc:482] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:11.278581 16208 raft_consensus.cc:504] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.280254 16208 leader_election.cc:288] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383), 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645)
I20260807 03:03:11.287050 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.280624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:34069 (local address 127.5.122.3:39383)
0807 03:03:11.280810 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.280814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.281669 (+ 855us) server_negotiation.cc:407] Connection header received
0807 03:03:11.281979 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.281983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.282042 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.282115 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:11.282634 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.283518 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.285347 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.286164 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.286273 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.286396 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.286683 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.286813 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:03:11.287163 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.280615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39383 (local address 127.5.122.2:34069)
0807 03:03:11.281344 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.281363 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.281820 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.282150 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.282155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.282183 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:11.282503 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.282512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.283642 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.283647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.285190 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.285202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.286368 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.286375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.286525 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.286559 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.286634 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.286756 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":1,"thread_start_us":374,"threads_started":1}
I20260807 03:03:11.287441 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.281549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:60303 (local address 127.5.122.1:43645)
0807 03:03:11.281748 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.281753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.282174 (+ 421us) server_negotiation.cc:407] Connection header received
0807 03:03:11.282410 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.282415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.282476 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.282547 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:11.283042 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.283947 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.285752 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.286585 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.286688 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.286781 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.287076 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.287197 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 03:03:11.287552 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.281411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43645 (local address 127.5.122.2:60303)
0807 03:03:11.281973 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:11.281990 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:11.282236 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:11.282581 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:11.282585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:11.282609 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:11.282906 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.282914 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.284133 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.284138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.285609 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:11.285621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.286790 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.286797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:11.286935 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.286958 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:03:11.287040 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 03:03:11.287164 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"spinlock_wait_cycles":6528,"thread_start_us":294,"threads_started":1}
I20260807 03:03:11.288087 16083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3932cc7a53974395ae6fa610d4ce390a" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" is_pre_election: true
I20260807 03:03:11.288597 16083 raft_consensus.cc:2454] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 0.
I20260807 03:03:11.288560 15675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3932cc7a53974395ae6fa610d4ce390a" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b066723907340d5b2b6cf63912ff335" is_pre_election: true
I20260807 03:03:11.289079 15675 raft_consensus.cc:2454] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 0.
I20260807 03:03:11.289206 15793 leader_election.cc:302] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 451bb126bdb647a48dd8b37ef76726a3, eefa93d1276e4bbbbe9d49ef727f9bad; no voters:
I20260807 03:03:11.289603 16208 raft_consensus.cc:2781] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:11.289734 16208 raft_consensus.cc:482] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:11.289835 16208 raft_consensus.cc:3037] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:11.292078 16208 raft_consensus.cc:504] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.292590 16208 leader_election.cc:288] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 election: Requested vote from peers eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383), 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645)
I20260807 03:03:11.292970 16083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3932cc7a53974395ae6fa610d4ce390a" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad"
I20260807 03:03:11.293100 15675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3932cc7a53974395ae6fa610d4ce390a" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b066723907340d5b2b6cf63912ff335"
I20260807 03:03:11.293174 16083 raft_consensus.cc:3037] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:11.293329 15675 raft_consensus.cc:3037] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:11.296360 16083 raft_consensus.cc:2454] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 1.
I20260807 03:03:11.296365 15675 raft_consensus.cc:2454] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 1.
I20260807 03:03:11.296715 15793 leader_election.cc:302] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 451bb126bdb647a48dd8b37ef76726a3, eefa93d1276e4bbbbe9d49ef727f9bad; no voters:
I20260807 03:03:11.296969 16208 raft_consensus.cc:2781] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:11.297365 16208 raft_consensus.cc:686] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 1 LEADER]: Becoming Leader. State: Replica: 451bb126bdb647a48dd8b37ef76726a3, State: Running, Role: LEADER
I20260807 03:03:11.297896 16208 consensus_queue.cc:237] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:11.302932 14388 catalog_manager.cc:5495] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2). New cstate: current_term: 1 leader_uuid: "451bb126bdb647a48dd8b37ef76726a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } health_report { overall_health: HEALTHY } } }
W20260807 03:03:11.303890 16149 tablet.cc:2369] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:03:11.331593 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.326788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36976 (local address 127.5.122.2:43873)
0807 03:03:11.326975 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.326981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.327103 (+ 122us) server_negotiation.cc:407] Connection header received
0807 03:03:11.327254 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.327258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.327386 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.327501 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:11.327775 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.328465 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.329656 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.330032 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.330115 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.330560 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:11.331031 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:11.331159 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.331269 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.331347 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 03:03:11.339004 15675 raft_consensus.cc:1261] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Refusing update from remote peer 451bb126bdb647a48dd8b37ef76726a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:11.339004 16083 raft_consensus.cc:1261] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Refusing update from remote peer 451bb126bdb647a48dd8b37ef76726a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:11.339892 16218 consensus_queue.cc:1035] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:11.340224 16208 consensus_queue.cc:1035] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 03:03:11.347447 15744 tablet.cc:2369] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:03:11.353636 16221 mvcc.cc:205] Tried to move back new op lower bound from 7315750057315217408 to 7315750057160065024. Current Snapshot: MvccSnapshot[applied={T|T < 7315750057315217408}]
I20260807 03:03:11.370558 14387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:11.370896 14387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:11.379333 14387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a8beeac975047818b5f8a6f109d4882: Sending DeleteTablet for 3 replicas of tablet 3932cc7a53974395ae6fa610d4ce390a
I20260807 03:03:11.380574 16054 tablet_service.cc:1514] Processing DeleteTablet for tablet 3932cc7a53974395ae6fa610d4ce390a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:11 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47778
I20260807 03:03:11.380638 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet 3932cc7a53974395ae6fa610d4ce390a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:11 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:36964
I20260807 03:03:11.381568 16237 tablet_replica.cc:330] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: stopping tablet replica
I20260807 03:03:11.381579 15648 tablet_service.cc:1514] Processing DeleteTablet for tablet 3932cc7a53974395ae6fa610d4ce390a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:11 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:33638
I20260807 03:03:11.382145 16237 raft_consensus.cc:2229] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:03:11.382980 16237 raft_consensus.cc:2258] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:11.387569 16238 tablet_replica.cc:330] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: stopping tablet replica
I20260807 03:03:11.388087 16238 raft_consensus.cc:2229] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:11.388659 16238 raft_consensus.cc:2258] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:11.389712 16240 tablet_replica.cc:330] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: stopping tablet replica
I20260807 03:03:11.390158 16240 raft_consensus.cc:2229] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:11.390548 16237 ts_tablet_manager.cc:1905] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:11.390816 16240 raft_consensus.cc:2258] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:11.392442 16238 ts_tablet_manager.cc:1905] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:11.399833 16238 ts_tablet_manager.cc:1918] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:11.400068 16238 log.cc:1196] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/3932cc7a53974395ae6fa610d4ce390a
I20260807 03:03:11.400681 16238 ts_tablet_manager.cc:1939] T 3932cc7a53974395ae6fa610d4ce390a P eefa93d1276e4bbbbe9d49ef727f9bad: Deleting consensus metadata
I20260807 03:03:11.400815 16240 ts_tablet_manager.cc:1905] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:11.402683 14372 catalog_manager.cc:4841] TS eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383): tablet 3932cc7a53974395ae6fa610d4ce390a (table test-table [id=51501770ec5a44dab2fb1a290680e19c]) successfully deleted
I20260807 03:03:11.405519 16237 ts_tablet_manager.cc:1918] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:11.405742 16237 log.cc:1196] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/3932cc7a53974395ae6fa610d4ce390a
I20260807 03:03:11.406404 16237 ts_tablet_manager.cc:1939] T 3932cc7a53974395ae6fa610d4ce390a P 451bb126bdb647a48dd8b37ef76726a3: Deleting consensus metadata
I20260807 03:03:11.408483 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 14347
I20260807 03:03:11.408740 14372 catalog_manager.cc:4841] TS 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2:43873): tablet 3932cc7a53974395ae6fa610d4ce390a (table test-table [id=51501770ec5a44dab2fb1a290680e19c]) successfully deleted
I20260807 03:03:11.409659 14432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:11.413021 16240 ts_tablet_manager.cc:1918] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:11.413219 16240 log.cc:1196] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/3932cc7a53974395ae6fa610d4ce390a
I20260807 03:03:11.413764 16240 ts_tablet_manager.cc:1939] T 3932cc7a53974395ae6fa610d4ce390a P 3b066723907340d5b2b6cf63912ff335: Deleting consensus metadata
I20260807 03:03:11.425426 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.387521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.5.122.61:45345)
0807 03:03:11.387691 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.387697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.407157 (+ 19460us) server_negotiation.cc:407] Connection header received
0807 03:03:11.407378 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.407384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.407474 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.407569 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.410972 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.412150 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.414151 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.414784 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.414919 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.418794 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.418801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.418806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.418856 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.422525 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.423384 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.423391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.423395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.423547 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.423979 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.423984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.423987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.424264 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.424545 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.424847 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.425068 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:03:11.438170 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.407427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.5.122.60:36569)
0807 03:03:11.407585 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.407590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.425569 (+ 17979us) server_negotiation.cc:407] Connection header received
0807 03:03:11.426624 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.426629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.426712 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.426778 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.428150 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.428872 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.430072 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.430547 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.430662 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.433047 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.433052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.433055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.433113 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.435575 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.436160 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.436165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.436169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.436308 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.436776 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.436781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.436783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.437048 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.437343 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.437588 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.437795 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 03:03:11.566691 14372 catalog_manager.cc:4841] TS 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645): tablet 3932cc7a53974395ae6fa610d4ce390a (table test-table [id=51501770ec5a44dab2fb1a290680e19c]) successfully deleted
W20260807 03:03:11.566946 14372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.122.1:43645, user_credentials={real_user=slave}} blocked reactor thread for 151323us
I20260807 03:03:11.567639 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 14347
I20260807 03:03:11.618283 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.600254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.5.122.61:45345)
0807 03:03:11.600478 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.600484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.601732 (+ 1248us) server_negotiation.cc:407] Connection header received
0807 03:03:11.602004 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.602010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.602088 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.602152 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.604247 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.605337 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.607484 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.608044 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.608137 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.611777 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.611784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.611789 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.611831 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.615160 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.616046 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.616051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.616055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.616214 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.616632 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.616637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.616640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.616987 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.617217 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.617679 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.617904 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:03:11.622242 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:11.601115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.5.122.60:36569)
0807 03:03:11.601294 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:11.601299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:11.607288 (+ 5989us) server_negotiation.cc:407] Connection header received
0807 03:03:11.608029 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:11.608033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:11.608100 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:11.608181 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:11.609934 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.610712 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.612075 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:11.612621 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:11.612709 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:11.616050 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:11.616057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:11.616061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:11.616100 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:11.619706 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.620428 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.620433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.620437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.620551 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:11.620896 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:11.620901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:11.620904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:11.621184 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:11.621423 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:11.621714 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 03:03:11.621924 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:03:12.360976 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.360143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:41037)
0807 03:03:12.360617 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.360766 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":242,"threads_started":1}
I20260807 03:03:12.361156 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.360240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.1:50463)
0807 03:03:12.360849 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.360990 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"threads_started":1}
W20260807 03:03:12.361690 15739 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:45857 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
W20260807 03:03:12.361748 15950 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:45857 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
I20260807 03:03:12.363317 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.362520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.3:59719)
0807 03:03:12.362972 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.363103 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":257,"threads_started":1}
W20260807 03:03:12.363713 16147 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:45857 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
I20260807 03:03:12.878625 16256 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1a8beeac975047818b5f8a6f109d4882)
I20260807 03:03:12.878809 16257 raft_consensus.cc:482] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1a8beeac975047818b5f8a6f109d4882)
I20260807 03:03:12.878921 16256 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } }
I20260807 03:03:12.879067 16257 raft_consensus.cc:504] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } }
I20260807 03:03:12.880380 16256 leader_election.cc:288] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a8beeac975047818b5f8a6f109d4882 (127.5.122.62:45857), f862fb4523e3488eada0c8a770e48fe1 (127.5.122.61:45345)
I20260807 03:03:12.880540 16257 leader_election.cc:288] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a8beeac975047818b5f8a6f109d4882 (127.5.122.62:45857), c7ab14c14ed04dfda8dffd59f0b5d10d (127.5.122.60:36569)
I20260807 03:03:12.881736 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.880520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46238)
0807 03:03:12.881145 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.881369 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"spinlock_wait_cycles":1280,"thread_start_us":453,"threads_started":1}
W20260807 03:03:12.881982 15464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
I20260807 03:03:12.890137 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.880523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:45345 (local address 127.0.0.1:32818)
0807 03:03:12.881932 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.881950 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:12.882090 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:12.882978 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:12.882985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:12.883015 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:12.883283 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.883294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.885181 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.885187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.887437 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.887455 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.889260 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.889267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.889489 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.889531 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 03:03:12.889605 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 03:03:12.889760 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"mutex_wait_us":163,"threads_started":1}
I20260807 03:03:12.891108 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.880790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.5.122.61:45345)
0807 03:03:12.882615 (+ 1825us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:12.882622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:12.882645 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:03:12.882717 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:12.882723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:12.882802 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:12.883419 (+ 617us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:12.883654 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.884918 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.887712 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.889034 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.890412 (+ 1378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.890529 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:12.890585 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 03:03:12.890759 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1693,"thread_start_us":1599,"threads_started":1}
I20260807 03:03:12.891969 15234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f862fb4523e3488eada0c8a770e48fe1" is_pre_election: true
I20260807 03:03:12.892364 15234 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ab14c14ed04dfda8dffd59f0b5d10d in term 1.
I20260807 03:03:12.892988 15467 leader_election.cc:302] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7ab14c14ed04dfda8dffd59f0b5d10d, f862fb4523e3488eada0c8a770e48fe1; no voters:
I20260807 03:03:12.893149 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.892500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46260)
0807 03:03:12.892657 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.892859 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 03:03:12.893422 16256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260807 03:03:12.893400 15464 leader_election.cc:334] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1a8beeac975047818b5f8a6f109d4882 (127.5.122.62:45857): Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
I20260807 03:03:12.893608 16256 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1a8beeac975047818b5f8a6f109d4882)
I20260807 03:03:12.893714 16256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 1 FOLLOWER]: Advancing to term 2
I20260807 03:03:12.893724 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.880481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46240)
0807 03:03:12.891927 (+ 11446us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.892127 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11322,"thread_start_us":11189,"threads_started":1}
W20260807 03:03:12.894021 15201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
I20260807 03:03:12.896121 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.895727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46270)
0807 03:03:12.895894 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.895979 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 03:03:12.896298 15201 leader_election.cc:334] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1a8beeac975047818b5f8a6f109d4882 (127.5.122.62:45857): Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
I20260807 03:03:12.897131 16256 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } }
I20260807 03:03:12.897894 16256 leader_election.cc:288] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [CANDIDATE]: Term 2 election: Requested vote from peers 1a8beeac975047818b5f8a6f109d4882 (127.5.122.62:45857), f862fb4523e3488eada0c8a770e48fe1 (127.5.122.61:45345)
I20260807 03:03:12.898540 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.898143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46288)
0807 03:03:12.898304 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.898416 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 03:03:12.898617 15234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f862fb4523e3488eada0c8a770e48fe1"
I20260807 03:03:12.898854 15234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 03:03:12.899698 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.899297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46294)
0807 03:03:12.899476 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.899554 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 03:03:12.899916 15464 leader_election.cc:334] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1a8beeac975047818b5f8a6f109d4882 (127.5.122.62:45857): Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
I20260807 03:03:12.901153 15234 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ab14c14ed04dfda8dffd59f0b5d10d in term 2.
I20260807 03:03:12.901319 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.892482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55688 (local address 127.5.122.60:36569)
0807 03:03:12.894151 (+ 1669us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:12.894157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:12.894179 (+ 22us) server_negotiation.cc:407] Connection header received
0807 03:03:12.894237 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:12.894241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:12.894348 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:12.894465 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:12.895248 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.896334 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.899312 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.900400 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.900518 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.900616 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:12.900885 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 03:03:12.901011 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1544,"thread_start_us":1757,"threads_started":1}
I20260807 03:03:12.901352 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.892128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:36569 (local address 127.0.0.1:55688)
0807 03:03:12.893072 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.893092 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:12.893359 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:12.894528 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:12.894534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:12.894569 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:12.894993 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.895005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.896504 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.896509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.899163 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.899179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.900581 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.900587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.900749 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.900773 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 03:03:12.900851 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 03:03:12.900990 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":1244,"threads_started":1}
I20260807 03:03:12.901576 15467 leader_election.cc:302] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7ab14c14ed04dfda8dffd59f0b5d10d, f862fb4523e3488eada0c8a770e48fe1; no voters: 1a8beeac975047818b5f8a6f109d4882
I20260807 03:03:12.901889 16256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 03:03:12.902127 15498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f862fb4523e3488eada0c8a770e48fe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" is_pre_election: true
I20260807 03:03:12.902360 16256 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [term 2 LEADER]: Becoming Leader. State: Replica: c7ab14c14ed04dfda8dffd59f0b5d10d, State: Running, Role: LEADER
I20260807 03:03:12.902716 16256 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } }
I20260807 03:03:12.904590 16270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7ab14c14ed04dfda8dffd59f0b5d10d. Latest consensus state: current_term: 2 leader_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:12.904942 15201 leader_election.cc:302] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f862fb4523e3488eada0c8a770e48fe1; no voters: 1a8beeac975047818b5f8a6f109d4882, c7ab14c14ed04dfda8dffd59f0b5d10d
I20260807 03:03:12.905033 16270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:12.905241 16257 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:03:12.905675 16273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:12.909677 16273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:12.910492 16273 catalog_manager.cc:1259] Loaded cluster ID: 7812f86194aa45bd8c28110b55a4fc03
I20260807 03:03:12.910563 16273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:12.911623 16273 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:12.912405 16273 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Loaded TSK: 0
I20260807 03:03:12.913168 16273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:03:12.956039 15483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:03:12.963281 15483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:12.967108 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.966726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46316)
0807 03:03:12.966890 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.966963 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 03:03:12.967348 15234 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [term 2 FOLLOWER]: Refusing update from remote peer c7ab14c14ed04dfda8dffd59f0b5d10d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 03:03:12.968017 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.967680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46322)
0807 03:03:12.967819 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.967887 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 03:03:12.968149 16270 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 03:03:12.968425 15464 consensus_peers.cc:487] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d -> Peer 1a8beeac975047818b5f8a6f109d4882 (127.5.122.62:45857): Couldn't send request to peer 1a8beeac975047818b5f8a6f109d4882. Status: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 03:03:12.970691 16257 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7ab14c14ed04dfda8dffd59f0b5d10d. Latest consensus state: current_term: 2 leader_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:12.971004 16257 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:03:12.972441 16270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:12.972754 16270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:12.972895 16276 mvcc.cc:205] Tried to move back new op lower bound from 7315750063987601408 to 7315750063733260288. Current Snapshot: MvccSnapshot[applied={T|T < 7315750063987601408}]
I20260807 03:03:12.974011 16257 sys_catalog.cc:455] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1a8beeac975047818b5f8a6f109d4882" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45857 } } peers { permanent_uuid: "f862fb4523e3488eada0c8a770e48fe1" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "c7ab14c14ed04dfda8dffd59f0b5d10d" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 36569 } attrs { promote: false } } }
I20260807 03:03:12.974260 16257 sys_catalog.cc:458] T 00000000000000000000000000000000 P f862fb4523e3488eada0c8a770e48fe1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:03:12.994513 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.984623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:43645 (local address 127.0.0.1:33648)
0807 03:03:12.984785 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.984806 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:12.984934 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:12.985833 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:12.985838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:12.985864 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:12.986221 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.986234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.987743 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.987750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.990029 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.990046 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.993565 (+ 3519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.993573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.993870 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.993910 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 03:03:12.993978 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 03:03:12.994117 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":1}
I20260807 03:03:12.995154 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.984908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39383 (local address 127.0.0.1:47780)
0807 03:03:12.985066 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.985083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:12.985180 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:12.986131 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:12.986137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:12.986166 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:12.986453 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.986465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.989071 (+ 2606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.989077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.991761 (+ 2684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.991776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.993430 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.993437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:12.993681 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.993720 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 03:03:12.993822 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 03:03:12.993980 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":683}
I20260807 03:03:12.995961 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.985266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47780 (local address 127.5.122.3:39383)
0807 03:03:12.985802 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:12.985807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:12.985825 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:03:12.985892 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:12.985896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:12.985983 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:12.986087 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:12.987755 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.988896 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.991973 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.993239 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.994244 (+ 1005us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.995439 (+ 1195us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:12.995505 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 03:03:12.995663 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":294,"threads_started":1}
I20260807 03:03:12.996044 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.984749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.5.122.1:43645)
0807 03:03:12.985465 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:12.985471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:12.985490 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:03:12.985561 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:12.985566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:12.985642 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:12.985864 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:12.986399 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.987577 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.992025 (+ 4448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:12.993294 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:12.995372 (+ 2078us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:12.995490 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:12.995547 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 03:03:12.995698 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":463,"threads_started":1}
I20260807 03:03:12.997002 15648 tablet_service.cc:1467] Processing CreateTablet for tablet 07f2cb8a0d1b4fdc857eebb910a50234 (DEFAULT_TABLE table=test-table [id=7d2849ab61034493942d8497ff200da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:12.997435 15648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f2cb8a0d1b4fdc857eebb910a50234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:13.002513 16290 tablet_bootstrap.cc:492] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: Bootstrap starting.
I20260807 03:03:13.002494 16054 tablet_service.cc:1467] Processing CreateTablet for tablet 07f2cb8a0d1b4fdc857eebb910a50234 (DEFAULT_TABLE table=test-table [id=7d2849ab61034493942d8497ff200da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:13.002959 16054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f2cb8a0d1b4fdc857eebb910a50234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:13.003247 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.985617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:43873 (local address 127.0.0.1:36988)
0807 03:03:12.994749 (+ 9132us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:12.994769 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:12.994877 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:12.998977 (+ 4100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:12.998981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:12.999004 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:12.999148 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:12.999159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.000321 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.000327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:13.001457 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:13.001470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.002531 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.002537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:13.002744 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:13.002768 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 03:03:13.002852 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 03:03:13.002970 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9039,"thread_start_us":10556,"threads_started":1}
I20260807 03:03:13.003405 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:12.986495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.5.122.2:43873)
0807 03:03:12.998680 (+ 12185us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:12.998684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:12.998699 (+ 15us) server_negotiation.cc:407] Connection header received
0807 03:03:12.998755 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:12.998758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:12.998821 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:12.998911 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:12.999291 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.000134 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.001618 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.002390 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.002510 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:13.002600 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:13.002899 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 03:03:13.003068 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12076,"thread_start_us":12532,"threads_started":1}
I20260807 03:03:13.004204 16290 tablet_bootstrap.cc:654] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:13.006412 16290 tablet_bootstrap.cc:492] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: No bootstrap required, opened a new log
I20260807 03:03:13.006680 16290 ts_tablet_manager.cc:1397] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260807 03:03:13.007169 15851 tablet_service.cc:1467] Processing CreateTablet for tablet 07f2cb8a0d1b4fdc857eebb910a50234 (DEFAULT_TABLE table=test-table [id=7d2849ab61034493942d8497ff200da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:13.007500 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f2cb8a0d1b4fdc857eebb910a50234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:13.007449 16290 raft_consensus.cc:348] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.007694 16290 raft_consensus.cc:374] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:13.007769 16290 raft_consensus.cc:729] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b066723907340d5b2b6cf63912ff335, State: Initialized, Role: FOLLOWER
I20260807 03:03:13.007999 16290 consensus_queue.cc:260] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.009617 16290 ts_tablet_manager.cc:1428] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 03:03:13.012842 16298 tablet_bootstrap.cc:492] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: Bootstrap starting.
I20260807 03:03:13.014564 16298 tablet_bootstrap.cc:654] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:13.015085 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.009033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.1:55133)
0807 03:03:13.014706 (+ 5673us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.014824 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5543,"thread_start_us":5440,"threads_started":1}
I20260807 03:03:13.015861 16299 tablet_bootstrap.cc:492] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: Bootstrap starting.
I20260807 03:03:13.016083 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.015663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.1:33381)
0807 03:03:13.015861 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.015953 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 03:03:13.017081 15738 heartbeater.cc:503] Master 127.5.122.60:36569 was elected leader, sending a full tablet report...
I20260807 03:03:13.017556 16299 tablet_bootstrap.cc:654] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:13.019397 16298 tablet_bootstrap.cc:492] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: No bootstrap required, opened a new log
I20260807 03:03:13.019513 16298 ts_tablet_manager.cc:1397] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 03:03:13.019574 16299 tablet_bootstrap.cc:492] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: No bootstrap required, opened a new log
I20260807 03:03:13.019711 16299 ts_tablet_manager.cc:1397] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 03:03:13.020160 16298 raft_consensus.cc:348] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.020339 16298 raft_consensus.cc:374] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:13.020427 16298 raft_consensus.cc:729] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451bb126bdb647a48dd8b37ef76726a3, State: Initialized, Role: FOLLOWER
I20260807 03:03:13.020345 16299 raft_consensus.cc:348] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.020555 16299 raft_consensus.cc:374] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:13.020642 16299 raft_consensus.cc:729] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eefa93d1276e4bbbbe9d49ef727f9bad, State: Initialized, Role: FOLLOWER
I20260807 03:03:13.020666 16298 consensus_queue.cc:260] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.020866 16299 consensus_queue.cc:260] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.021414 16298 ts_tablet_manager.cc:1428] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 03:03:13.022722 16299 ts_tablet_manager.cc:1428] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 03:03:13.026968 16146 heartbeater.cc:503] Master 127.5.122.60:36569 was elected leader, sending a full tablet report...
I20260807 03:03:13.027287 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.023339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.3:43235)
0807 03:03:13.026891 (+ 3552us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.027026 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3406,"thread_start_us":3217,"threads_started":1}
I20260807 03:03:13.028679 15949 heartbeater.cc:503] Master 127.5.122.60:36569 was elected leader, sending a full tablet report...
I20260807 03:03:13.028968 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.022007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:50281)
0807 03:03:13.028560 (+ 6553us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.028704 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6436,"thread_start_us":6303,"threads_started":1}
I20260807 03:03:13.029479 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.029125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.3:54849)
0807 03:03:13.029264 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.029345 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 03:03:13.030037 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.029761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:49935)
0807 03:03:13.029882 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.029945 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 03:03:13.137126 16305 raft_consensus.cc:482] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:13.137390 16305 raft_consensus.cc:504] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.138062 16305 leader_election.cc:288] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645), eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383)
I20260807 03:03:13.138621 15675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2cb8a0d1b4fdc857eebb910a50234" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b066723907340d5b2b6cf63912ff335" is_pre_election: true
I20260807 03:03:13.138680 16083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2cb8a0d1b4fdc857eebb910a50234" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" is_pre_election: true
I20260807 03:03:13.138887 15675 raft_consensus.cc:2454] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 0.
I20260807 03:03:13.138922 16083 raft_consensus.cc:2454] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 0.
I20260807 03:03:13.139372 15793 leader_election.cc:302] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b066723907340d5b2b6cf63912ff335, 451bb126bdb647a48dd8b37ef76726a3; no voters:
I20260807 03:03:13.139622 16305 raft_consensus.cc:2781] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:13.139724 16305 raft_consensus.cc:482] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:13.139814 16305 raft_consensus.cc:3037] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:13.141865 16305 raft_consensus.cc:504] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.142388 16305 leader_election.cc:288] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 election: Requested vote from peers 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645), eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383)
I20260807 03:03:13.142793 15675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2cb8a0d1b4fdc857eebb910a50234" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b066723907340d5b2b6cf63912ff335"
I20260807 03:03:13.142876 16083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2cb8a0d1b4fdc857eebb910a50234" candidate_uuid: "451bb126bdb647a48dd8b37ef76726a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad"
I20260807 03:03:13.142976 15675 raft_consensus.cc:3037] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:13.143057 16083 raft_consensus.cc:3037] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:13.144966 16083 raft_consensus.cc:2454] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 1.
I20260807 03:03:13.145174 15675 raft_consensus.cc:2454] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 451bb126bdb647a48dd8b37ef76726a3 in term 1.
I20260807 03:03:13.145287 15793 leader_election.cc:302] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 451bb126bdb647a48dd8b37ef76726a3, eefa93d1276e4bbbbe9d49ef727f9bad; no voters:
I20260807 03:03:13.145555 16305 raft_consensus.cc:2781] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:13.145973 16305 raft_consensus.cc:686] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 1 LEADER]: Becoming Leader. State: Replica: 451bb126bdb647a48dd8b37ef76726a3, State: Running, Role: LEADER
I20260807 03:03:13.146246 16305 consensus_queue.cc:237] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } }
I20260807 03:03:13.147044 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.146551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:41263)
0807 03:03:13.146723 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.146838 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 03:03:13.148936 15483 catalog_manager.cc:5495] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2). New cstate: current_term: 1 leader_uuid: "451bb126bdb647a48dd8b37ef76726a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "451bb126bdb647a48dd8b37ef76726a3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 43873 } health_report { overall_health: HEALTHY } } }
I20260807 03:03:13.180719 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.176767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.5.122.2:43873)
0807 03:03:13.176956 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:13.176960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:13.176976 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:03:13.177082 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:13.177086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:13.177145 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:13.177214 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:13.177518 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.178234 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.179429 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.179812 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.179896 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:13.180186 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:13.180274 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:13.180333 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:13.180429 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 03:03:13.180486 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 03:03:13.183514 15675 raft_consensus.cc:1261] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Refusing update from remote peer 451bb126bdb647a48dd8b37ef76726a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:13.183573 16083 raft_consensus.cc:1261] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Refusing update from remote peer 451bb126bdb647a48dd8b37ef76726a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:13.184087 16310 consensus_queue.cc:1035] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b066723907340d5b2b6cf63912ff335" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 43645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:13.184397 16305 consensus_queue.cc:1035] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eefa93d1276e4bbbbe9d49ef727f9bad" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:13.186878 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.186468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.1:38321)
0807 03:03:13.186599 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.186702 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 03:03:13.187328 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.186844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.3:54037)
0807 03:03:13.187001 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.187131 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 03:03:13.191643 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.191159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:48317)
0807 03:03:13.191394 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.191479 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 03:03:13.192582 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.192159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.3:43659)
0807 03:03:13.192361 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.192443 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 03:03:13.198904 15483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:13.199179 15483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:13.204666 15483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7ab14c14ed04dfda8dffd59f0b5d10d: Sending DeleteTablet for 3 replicas of tablet 07f2cb8a0d1b4fdc857eebb910a50234
I20260807 03:03:13.205554 15648 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f2cb8a0d1b4fdc857eebb910a50234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:13 UTC) from {username='oryx', principal='oryx/127.5.122.60@KRBTEST.COM'} at 127.0.0.1:33648
I20260807 03:03:13.205639 16054 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f2cb8a0d1b4fdc857eebb910a50234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:13 UTC) from {username='oryx', principal='oryx/127.5.122.60@KRBTEST.COM'} at 127.0.0.1:47780
I20260807 03:03:13.206193 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f2cb8a0d1b4fdc857eebb910a50234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:13 UTC) from {username='oryx', principal='oryx/127.5.122.60@KRBTEST.COM'} at 127.0.0.1:36988
I20260807 03:03:13.207882 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 15487
I20260807 03:03:13.211017 16327 tablet_replica.cc:330] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: stopping tablet replica
I20260807 03:03:13.211223 16327 raft_consensus.cc:2229] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:03:13.211683 16327 raft_consensus.cc:2258] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:13.212360 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.211886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.2:51169)
0807 03:03:13.212062 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.212161 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 03:03:13.215005 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.214534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.5.122.1:51439)
0807 03:03:13.214751 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.214837 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 03:03:13.216427 16326 tablet_replica.cc:330] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: stopping tablet replica
I20260807 03:03:13.216610 16326 raft_consensus.cc:2229] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:13.216683 16327 ts_tablet_manager.cc:1905] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:13.216915 16326 raft_consensus.cc:2258] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:13.218933 16325 tablet_replica.cc:330] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: stopping tablet replica
I20260807 03:03:13.219111 16325 raft_consensus.cc:2229] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:13.219376 16325 raft_consensus.cc:2258] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:13.221658 16326 ts_tablet_manager.cc:1905] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:13.222074 16325 ts_tablet_manager.cc:1905] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:13.224212 16327 ts_tablet_manager.cc:1918] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:13.224352 16327 log.cc:1196] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/07f2cb8a0d1b4fdc857eebb910a50234
I20260807 03:03:13.224711 16327 ts_tablet_manager.cc:1939] T 07f2cb8a0d1b4fdc857eebb910a50234 P 451bb126bdb647a48dd8b37ef76726a3: Deleting consensus metadata
I20260807 03:03:13.225682 15468 catalog_manager.cc:4841] TS 451bb126bdb647a48dd8b37ef76726a3 (127.5.122.2:43873): tablet 07f2cb8a0d1b4fdc857eebb910a50234 (table test-table [id=7d2849ab61034493942d8497ff200da1]) successfully deleted
I20260807 03:03:13.227815 16326 ts_tablet_manager.cc:1918] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:13.227950 16326 log.cc:1196] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/07f2cb8a0d1b4fdc857eebb910a50234
I20260807 03:03:13.228363 16326 ts_tablet_manager.cc:1939] T 07f2cb8a0d1b4fdc857eebb910a50234 P eefa93d1276e4bbbbe9d49ef727f9bad: Deleting consensus metadata
I20260807 03:03:13.229343 15468 catalog_manager.cc:4841] TS eefa93d1276e4bbbbe9d49ef727f9bad (127.5.122.3:39383): tablet 07f2cb8a0d1b4fdc857eebb910a50234 (table test-table [id=7d2849ab61034493942d8497ff200da1]) successfully deleted
I20260807 03:03:13.229991 16325 ts_tablet_manager.cc:1918] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:13.230114 16325 log.cc:1196] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/07f2cb8a0d1b4fdc857eebb910a50234
I20260807 03:03:13.230463 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.210004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.5.122.1:43645)
0807 03:03:13.210231 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:13.210238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:13.213345 (+ 3107us) server_negotiation.cc:407] Connection header received
0807 03:03:13.213587 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:13.213593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:13.213708 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:13.213783 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:13.217213 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.218383 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.220098 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.220755 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.220867 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:13.224070 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:13.224085 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:13.224104 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:13.224139 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:13.227482 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:13.228213 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:13.228221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:13.228223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:13.228384 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:13.228779 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:13.228783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:13.228785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:13.229147 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:13.229461 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:13.229868 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 03:03:13.230107 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 03:03:13.230590 16325 ts_tablet_manager.cc:1939] T 07f2cb8a0d1b4fdc857eebb910a50234 P 3b066723907340d5b2b6cf63912ff335: Deleting consensus metadata
I20260807 03:03:13.231072 15731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:13.231489 15468 catalog_manager.cc:4841] TS 3b066723907340d5b2b6cf63912ff335 (127.5.122.1:43645): tablet 07f2cb8a0d1b4fdc857eebb910a50234 (table test-table [id=7d2849ab61034493942d8497ff200da1]) successfully deleted
I20260807 03:03:13.324792 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 15487
I20260807 03:03:13.344426 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 15758
I20260807 03:03:13.357718 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.345214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.5.122.2:43873)
0807 03:03:13.345410 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:13.345416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:13.345457 (+ 41us) server_negotiation.cc:407] Connection header received
0807 03:03:13.345655 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:13.345659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:13.345726 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:13.345810 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:13.347255 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.348015 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.349357 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.349759 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.349870 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:13.352334 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:13.352350 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:13.352366 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:13.352407 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:13.355150 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:13.355772 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:13.355780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:13.355782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:13.355913 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:13.356269 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:13.356273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:13.356275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:13.356629 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:13.356927 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:13.357201 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 03:03:13.357416 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 03:03:13.358417 15942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:13.491004 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 15758
I20260807 03:03:13.518821 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 15965
I20260807 03:03:13.531425 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.519566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.5.122.3:39383)
0807 03:03:13.520007 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:13.520012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:13.520028 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:03:13.520084 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:13.520088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:13.520154 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:13.520241 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:13.521492 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.522188 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.523371 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:13.523902 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:13.524002 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:13.526447 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:13.526465 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:13.526482 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:13.526522 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:13.529201 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:13.529759 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:13.529765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:13.529768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:13.529899 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:13.530170 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:13.530173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:13.530175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:13.530562 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:13.530790 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:13.531021 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 03:03:13.531182 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":211,"threads_started":1}
I20260807 03:03:13.532018 16140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:13.553308 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.552632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46338)
0807 03:03:13.553063 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.553149 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":221,"threads_started":1}
I20260807 03:03:13.554562 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:13.554219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:45857 (local address 127.0.0.1:46352)
0807 03:03:13.554386 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:13.554455 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 03:03:13.622555 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 15965
I20260807 03:03:13.650336 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 15024
I20260807 03:03:13.651427 15266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:13.768399 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 15024
I20260807 03:03:13.793828 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 15173
I20260807 03:03:13.794915 15528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:13.913059 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 15173
2026-08-07T03:03:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9486 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:03:14 dist-test-slave-fgb3 krb5kdc[16347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:03:14 dist-test-slave-fgb3 krb5kdc[16347](info): set up 2 sockets
Aug 07 03:03:14 dist-test-slave-fgb3 krb5kdc[16347](info): commencing operation
krb5kdc: starting...
W20260807 03:03:15.105654 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:03:15 dist-test-slave-fgb3 krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab.
Entry for principal oryx/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab.
I20260807 03:03:15.333328 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42317
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab
--principal=oryx/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:42317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:15.648326 16367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:15.648636 16367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:15.648761 16367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:15.655896 16367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:15.656014 16367 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 03:03:15.656078 16367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:15.656162 16367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:15.656226 16367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:15.656312 16367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:15.666029 16367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:42317
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.62.keytab
--principal=oryx/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:15.667481 16367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:15.668828 16367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:15.679847 16374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:15.679864 16375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:15.680027 16377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:15.681038 16367 server_base.cc:1034] running on GCE node
I20260807 03:03:15.681766 16367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:15.683178 16367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:15.684387 16367 hybrid_clock.cc:648] HybridClock initialized: now 1786071795684328 us; error 40 us; skew 500 ppm
I20260807 03:03:15.684693 16367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:15 dist-test-slave-fgb3 krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071795, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:15.687849 16367 init.cc:377] Logged in from keytab as oryx/127.5.122.62@KRBTEST.COM (short username oryx)
I20260807 03:03:15.689253 16367 webserver.cc:460] Webserver started at http://127.5.122.62:37395/ using document root <none> and password file <none>
I20260807 03:03:15.689934 16367 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:15.690032 16367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:15.690382 16367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:15.693272 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "a7bb971ca50546deb880a96b18417960"
format_stamp: "Formatted at 2026-08-07 03:03:15 on dist-test-slave-fgb3"
server_key: "b9e56cc2420eb8387593c53c49f8ee99"
server_key_iv: "646235bee888e4d69a4924ee614e4afa"
server_key_version: "encryptionkey@0"
I20260807 03:03:15.694067 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "a7bb971ca50546deb880a96b18417960"
format_stamp: "Formatted at 2026-08-07 03:03:15 on dist-test-slave-fgb3"
server_key: "b9e56cc2420eb8387593c53c49f8ee99"
server_key_iv: "646235bee888e4d69a4924ee614e4afa"
server_key_version: "encryptionkey@0"
I20260807 03:03:15.699045 16367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 03:03:15.702252 16392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:15.703527 16367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:03:15.703681 16367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "a7bb971ca50546deb880a96b18417960"
format_stamp: "Formatted at 2026-08-07 03:03:15 on dist-test-slave-fgb3"
server_key: "b9e56cc2420eb8387593c53c49f8ee99"
server_key_iv: "646235bee888e4d69a4924ee614e4afa"
server_key_version: "encryptionkey@0"
I20260807 03:03:15.703871 16367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:15.714877 16367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:15.718564 16367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:15.718789 16367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:15.738492 16367 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:42317
I20260807 03:03:15.738510 16479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:42317 every 8 connection(s)
I20260807 03:03:15.739763 16367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:15.743273 16481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:15.744949 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 16367
I20260807 03:03:15.745167 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:03:15.745972 5608 external_mini_cluster.cc:1428] Setting key 93cf46e8682492125fb9ef1663d2c4b3
I20260807 03:03:15.755124 16481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Bootstrap starting.
I20260807 03:03:15.758754 16481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Neither blocks nor log segments found. Creating new log.
Aug 07 03:03:15 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:15.759845 16481 log.cc:824] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:15.762648 16481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: No bootstrap required, opened a new log
I20260807 03:03:15.767515 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:15.747916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.5.122.62:42317)
0807 03:03:15.748695 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:15.748709 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:15.748748 (+ 39us) server_negotiation.cc:407] Connection header received
0807 03:03:15.750432 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:15.750452 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:15.750789 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:15.751076 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:15.752786 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:15.754032 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:15.755693 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:15.756315 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:15.756456 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:15.760849 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:15.760857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:15.760871 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:15.760906 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:15.763766 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:15.764371 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:15.764378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:15.764383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:15.764563 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:15.764938 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:15.764945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:15.764947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:15.765442 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:15.765737 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:15.766125 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 03:03:15.766556 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":371,"threads_started":1}
I20260807 03:03:15.770128 16481 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bb971ca50546deb880a96b18417960" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42317 } }
I20260807 03:03:15.770511 16481 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:15.770601 16481 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7bb971ca50546deb880a96b18417960, State: Initialized, Role: FOLLOWER
I20260807 03:03:15.771070 16481 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bb971ca50546deb880a96b18417960" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42317 } }
I20260807 03:03:15.771278 16481 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:15.771409 16481 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:15.771557 16481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:15.773722 16481 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bb971ca50546deb880a96b18417960" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42317 } }
I20260807 03:03:15.774228 16481 leader_election.cc:302] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7bb971ca50546deb880a96b18417960; no voters:
I20260807 03:03:15.774843 16481 leader_election.cc:288] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:03:15.774986 16488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:15.775604 16488 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [term 1 LEADER]: Becoming Leader. State: Replica: a7bb971ca50546deb880a96b18417960, State: Running, Role: LEADER
I20260807 03:03:15.776087 16488 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bb971ca50546deb880a96b18417960" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42317 } }
I20260807 03:03:15.776710 16481 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:15.778682 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7bb971ca50546deb880a96b18417960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bb971ca50546deb880a96b18417960" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42317 } } }
I20260807 03:03:15.778671 16491 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7bb971ca50546deb880a96b18417960. Latest consensus state: current_term: 1 leader_uuid: "a7bb971ca50546deb880a96b18417960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bb971ca50546deb880a96b18417960" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42317 } } }
I20260807 03:03:15.779131 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:15.779132 16491 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:15.782943 16494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:15.788790 16494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:15.796984 16494 catalog_manager.cc:1347] Generated new cluster ID: 4d12c8dda23e4a53bc20a0a43b3ce72d
I20260807 03:03:15.797089 16494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:15.806257 16494 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:03:15.807580 16494 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:15.816479 16494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Generated new TSK 0
I20260807 03:03:15.817198 16494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab.
Entry for principal oryx/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab.
I20260807 03:03:15.871618 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42317
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab
--principal=oryx/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:16.190826 16524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:16.191077 16524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:16.191213 16524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:16.198227 16524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:16.198350 16524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:16.198475 16524 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:03:16.198577 16524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:16.208102 16524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab
--principal=oryx/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:42317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:16.209280 16524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:16.210587 16524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:16.222765 16535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:16.222880 16537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:16.222882 16534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:16.223656 16524 server_base.cc:1034] running on GCE node
I20260807 03:03:16.224196 16524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:16.224998 16524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:16.226204 16524 hybrid_clock.cc:648] HybridClock initialized: now 1786071796226145 us; error 33 us; skew 500 ppm
I20260807 03:03:16.226492 16524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:16 dist-test-slave-fgb3 krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071796, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:16.229364 16524 init.cc:377] Logged in from keytab as oryx/127.5.122.1@KRBTEST.COM (short username oryx)
I20260807 03:03:16.230686 16524 webserver.cc:460] Webserver started at http://127.5.122.1:46223/ using document root <none> and password file <none>
I20260807 03:03:16.231294 16524 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:16.231388 16524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:16.231675 16524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:16.234434 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "c0c9ed81b70b4905876c97d5e99c5828"
format_stamp: "Formatted at 2026-08-07 03:03:16 on dist-test-slave-fgb3"
server_key: "90cf2a332ed3038677ae63f1d53d0873"
server_key_iv: "aa47c7e543789e7751c1ba8bac0c2bee"
server_key_version: "encryptionkey@0"
I20260807 03:03:16.235175 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "c0c9ed81b70b4905876c97d5e99c5828"
format_stamp: "Formatted at 2026-08-07 03:03:16 on dist-test-slave-fgb3"
server_key: "90cf2a332ed3038677ae63f1d53d0873"
server_key_iv: "aa47c7e543789e7751c1ba8bac0c2bee"
server_key_version: "encryptionkey@0"
I20260807 03:03:16.239751 16524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:03:16.242923 16556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:16.244045 16524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:03:16.244201 16524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "c0c9ed81b70b4905876c97d5e99c5828"
format_stamp: "Formatted at 2026-08-07 03:03:16 on dist-test-slave-fgb3"
server_key: "90cf2a332ed3038677ae63f1d53d0873"
server_key_iv: "aa47c7e543789e7751c1ba8bac0c2bee"
server_key_version: "encryptionkey@0"
I20260807 03:03:16.244402 16524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:16.270215 16524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:16.274077 16524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:16.274340 16524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:16.276898 16524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:16.276997 16524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:16.277149 16524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:16.277217 16524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:03:16 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071796, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:16.300660 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.277261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:55633 (local address 127.5.122.62:42317)
0807 03:03:16.278117 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:16.278121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:16.279802 (+ 1681us) server_negotiation.cc:407] Connection header received
0807 03:03:16.281902 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:16.281945 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:16.282043 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:16.282171 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:16.284078 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.285173 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.288116 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.288742 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.288869 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.293740 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:16.293748 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:16.293753 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:16.293798 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:16.297248 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.298023 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.298030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.298034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.298189 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.298750 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.298754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.298756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.299094 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:16.299239 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:16.300096 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 03:03:16.300307 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":665,"threads_started":1}
I20260807 03:03:16.301817 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.278891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42317 (local address 127.5.122.1:55633)
0807 03:03:16.279665 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:16.279700 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:16.281587 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:16.282449 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:16.282459 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:16.283109 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:16.283847 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.283868 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.285399 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.285409 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.287897 (+ 2488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.287916 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.289164 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.289174 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.289511 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.290581 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:16.290616 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:16.293513 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:16.297378 (+ 3865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.297384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.297394 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.297859 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.298303 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.298308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.298327 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.298625 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.299272 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:16.299279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:16.299725 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 03:03:16.299981 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 03:03:16.300442 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":347,"threads_started":1}
I20260807 03:03:16.319761 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.306959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42317 (local address 127.5.122.1:52793)
0807 03:03:16.307576 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:16.307591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:16.307689 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:16.307964 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:16.307967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:16.308418 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:16.308682 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.308690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.310029 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.310032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.311351 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.311368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.312359 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.312366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.312488 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.312989 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:16.313005 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:16.313645 (+ 640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:16.317365 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.317368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.317371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.317731 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.318086 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.318089 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.318092 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.318195 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.318990 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:16.318993 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:16.319137 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 03:03:16.319240 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 03:03:16.319450 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":323,"threads_started":1}
I20260807 03:03:16.319921 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.306905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:52793 (local address 127.5.122.62:42317)
0807 03:03:16.307085 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:16.307091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:16.307654 (+ 563us) server_negotiation.cc:407] Connection header received
0807 03:03:16.307777 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:16.307782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:16.307857 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:16.307956 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:16.308792 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.309913 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.311525 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.312167 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.312290 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.313807 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:16.313813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:16.313819 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:16.313854 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:16.317245 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.317825 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.317830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.317834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.317996 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.318534 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.318540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.318544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.318889 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:16.319040 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:16.319350 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 03:03:16.319580 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:03:16.330705 16524 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:36523
I20260807 03:03:16.330734 16729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:36523 every 8 connection(s)
I20260807 03:03:16.331761 16524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:16.333624 16731 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42317
I20260807 03:03:16.333717 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 16524
I20260807 03:03:16.333856 16731 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:16.333905 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:03:16.334487 16731 heartbeater.cc:511] Master 127.5.122.62:42317 requested a full tablet report, sending...
I20260807 03:03:16.334559 5608 external_mini_cluster.cc:1428] Setting key bae5001904f929ac5d8449dbff172259
I20260807 03:03:16.336464 16414 ts_manager.cc:194] Registered new tserver with Master: c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523)
I20260807 03:03:16.337795 16414 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:55633
WARNING: no policy specified for oryx/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab.
Entry for principal oryx/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab.
I20260807 03:03:16.383682 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42317
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab
--principal=oryx/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:16.709609 16736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:16.709882 16736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:16.710031 16736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:16.717247 16736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:16.717366 16736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:16.717509 16736 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:03:16.717625 16736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:16.727478 16736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab
--principal=oryx/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:42317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:16.728700 16736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:16.730046 16736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:16.743261 16745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:16.743232 16742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:16.743427 16741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:16.744158 16736 server_base.cc:1034] running on GCE node
I20260807 03:03:16.744705 16736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:16.745573 16736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:16.746783 16736 hybrid_clock.cc:648] HybridClock initialized: now 1786071796746703 us; error 59 us; skew 500 ppm
I20260807 03:03:16.747067 16736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:16 dist-test-slave-fgb3 krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071796, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:16.750605 16736 init.cc:377] Logged in from keytab as oryx/127.5.122.2@KRBTEST.COM (short username oryx)
I20260807 03:03:16.752331 16736 webserver.cc:460] Webserver started at http://127.5.122.2:40923/ using document root <none> and password file <none>
I20260807 03:03:16.753165 16736 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:16.753293 16736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:16.753656 16736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:16.757220 16736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "06dd9008e4be471389f553e7a04edb7b"
format_stamp: "Formatted at 2026-08-07 03:03:16 on dist-test-slave-fgb3"
server_key: "59a837c54ba842deb6baef96e9115f17"
server_key_iv: "88ba28e231aa1215240a506abc759837"
server_key_version: "encryptionkey@0"
I20260807 03:03:16.758031 16736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "06dd9008e4be471389f553e7a04edb7b"
format_stamp: "Formatted at 2026-08-07 03:03:16 on dist-test-slave-fgb3"
server_key: "59a837c54ba842deb6baef96e9115f17"
server_key_iv: "88ba28e231aa1215240a506abc759837"
server_key_version: "encryptionkey@0"
I20260807 03:03:16.763095 16736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 03:03:16.766532 16753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:16.767820 16736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:03:16.767974 16736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "06dd9008e4be471389f553e7a04edb7b"
format_stamp: "Formatted at 2026-08-07 03:03:16 on dist-test-slave-fgb3"
server_key: "59a837c54ba842deb6baef96e9115f17"
server_key_iv: "88ba28e231aa1215240a506abc759837"
server_key_version: "encryptionkey@0"
I20260807 03:03:16.768196 16736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:16.797359 16736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:16.801918 16736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:16.802209 16736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:16.805097 16736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:16.805209 16736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:16.805378 16736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:16.805464 16736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:03:16 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071796, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:16.829002 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.806606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42317 (local address 127.5.122.2:55323)
0807 03:03:16.807517 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:16.807565 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:16.809589 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:16.810384 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:16.810397 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:16.811121 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:16.811909 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.811932 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.813216 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.813220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.815275 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.815288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.816189 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.816199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.816545 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.817383 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:16.817422 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:16.821050 (+ 3628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:16.824407 (+ 3357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.824417 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.824435 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.825051 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.825422 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.825427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.825431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.825626 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.826169 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:16.826177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:16.826578 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 03:03:16.826934 (+ 356us) client_negotiation.cc:240] Negotiation successful
0807 03:03:16.827482 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":381,"threads_started":1}
I20260807 03:03:16.829936 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.805752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:55323 (local address 127.5.122.62:42317)
0807 03:03:16.805965 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:16.805970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:16.807679 (+ 1709us) server_negotiation.cc:407] Connection header received
0807 03:03:16.809849 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:16.809855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:16.809972 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:16.810056 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:16.812083 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.813060 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.815431 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.815905 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.816031 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.821290 (+ 5259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:16.821298 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:16.821303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:16.821348 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:16.824268 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.825172 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.825178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.825182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.825319 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.825741 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.825746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.825748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.826030 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:16.826172 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:16.829549 (+ 3377us) server_negotiation.cc:299] Negotiation successful
0807 03:03:16.829717 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 03:03:16.847990 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.835017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:40787 (local address 127.5.122.62:42317)
0807 03:03:16.835194 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:16.835200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:16.835728 (+ 528us) server_negotiation.cc:407] Connection header received
0807 03:03:16.835851 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:16.835857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:16.835955 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:16.836067 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:16.836890 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.838041 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.839571 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.840116 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.840245 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.842827 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:16.842834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:16.842839 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:16.842875 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:16.845217 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.846013 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.846017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.846019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.846137 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:16.846504 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:16.846507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:16.846509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:16.846818 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:16.846927 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:16.847435 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 03:03:16.847661 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:03:16.848312 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:16.835039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42317 (local address 127.5.122.2:40787)
0807 03:03:16.835646 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:16.835661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:16.835763 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:16.836071 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:16.836074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:16.836494 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:16.836767 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.836776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.838146 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.838149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.839421 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:16.839433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:16.840340 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:16.840348 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:16.840578 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:16.841233 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:16.841260 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:16.842617 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:16.845339 (+ 2722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.845345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.845350 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.845916 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.846227 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:16.846232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:16.846236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:16.846407 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:16.846919 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:16.846925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:16.847149 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 03:03:16.847315 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 03:03:16.847676 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":311,"threads_started":1}
I20260807 03:03:16.857017 16736 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:44659
I20260807 03:03:16.857046 16936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:44659 every 8 connection(s)
I20260807 03:03:16.858196 16736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:16.860150 16937 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42317
I20260807 03:03:16.860337 16937 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:16.860906 16937 heartbeater.cc:511] Master 127.5.122.62:42317 requested a full tablet report, sending...
I20260807 03:03:16.862174 16414 ts_manager.cc:194] Registered new tserver with Master: 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659)
I20260807 03:03:16.863015 16414 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:55323
I20260807 03:03:16.865796 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 16736
I20260807 03:03:16.865957 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:03:16.866447 5608 external_mini_cluster.cc:1428] Setting key 73821def618268f49c90c5bcc33b753d
WARNING: no policy specified for oryx/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab.
Entry for principal oryx/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab.
I20260807 03:03:16.919806 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42317
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab
--principal=oryx/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:17.238216 16946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:17.238605 16946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:17.238754 16946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:17.245800 16946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:17.245894 16946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:17.246016 16946 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 03:03:17.246126 16946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:17.255797 16946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab
--principal=oryx/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.62:42317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:17.256935 16946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:17.258183 16946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:17.270112 16951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:17.270061 16956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:17.270089 16952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:17.271204 16946 server_base.cc:1034] running on GCE node
I20260807 03:03:17.271788 16946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:17.272698 16946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:17.273913 16946 hybrid_clock.cc:648] HybridClock initialized: now 1786071797273834 us; error 54 us; skew 500 ppm
I20260807 03:03:17.274188 16946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:17 dist-test-slave-fgb3 krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071797, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:17.277143 16946 init.cc:377] Logged in from keytab as oryx/127.5.122.3@KRBTEST.COM (short username oryx)
I20260807 03:03:17.278589 16946 webserver.cc:460] Webserver started at http://127.5.122.3:39981/ using document root <none> and password file <none>
I20260807 03:03:17.279222 16946 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:17.279317 16946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:17.279613 16946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:17.282279 16946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1"
format_stamp: "Formatted at 2026-08-07 03:03:17 on dist-test-slave-fgb3"
server_key: "87d8fbf48c1c600eb6246a0016712fc0"
server_key_iv: "9cf986e1524b2d764bd081896be10680"
server_key_version: "encryptionkey@0"
I20260807 03:03:17.283095 16946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1"
format_stamp: "Formatted at 2026-08-07 03:03:17 on dist-test-slave-fgb3"
server_key: "87d8fbf48c1c600eb6246a0016712fc0"
server_key_iv: "9cf986e1524b2d764bd081896be10680"
server_key_version: "encryptionkey@0"
I20260807 03:03:17.287717 16946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:03:17.290953 16968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:17.292171 16946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:17.292311 16946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1"
format_stamp: "Formatted at 2026-08-07 03:03:17 on dist-test-slave-fgb3"
server_key: "87d8fbf48c1c600eb6246a0016712fc0"
server_key_iv: "9cf986e1524b2d764bd081896be10680"
server_key_version: "encryptionkey@0"
I20260807 03:03:17.292508 16946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:17.326066 16946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:17.330098 16946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:17.330430 16946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:17.333323 16946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:17.333431 16946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:17.333580 16946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:17.333714 16946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 03:03:17.340302 16731 heartbeater.cc:503] Master 127.5.122.62:42317 was elected leader, sending a full tablet report...
Aug 07 03:03:17 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071797, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:17.356033 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.334085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:34223 (local address 127.5.122.62:42317)
0807 03:03:17.334275 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.334281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.335883 (+ 1602us) server_negotiation.cc:407] Connection header received
0807 03:03:17.338094 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.338100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.338196 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.338309 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:17.339984 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.341024 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.343578 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.344258 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.344391 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.348631 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:17.348638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:17.348643 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:17.348682 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:17.352303 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:17.353293 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:17.353299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:17.353303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:17.353479 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:17.353921 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:17.353926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:17.353930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:17.354303 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:17.354508 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.355436 (+ 928us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.355687 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 03:03:17.357041 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.334766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42317 (local address 127.5.122.3:34223)
0807 03:03:17.335701 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.335753 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.337759 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.338565 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.338572 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.339080 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:17.339742 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.339774 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.341242 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.341247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.343402 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.343427 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.344505 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.344513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.344811 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.345501 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:17.345522 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:17.348378 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:17.352508 (+ 4130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:17.352518 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:17.352538 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:17.353157 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:17.353593 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:17.353597 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:17.353599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:17.353812 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:17.354478 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:17.354486 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:17.354994 (+ 508us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.355295 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.355835 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":404,"threads_started":1}
I20260807 03:03:17.378299 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.363356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:48841 (local address 127.5.122.62:42317)
0807 03:03:17.363535 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.363541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.364478 (+ 937us) server_negotiation.cc:407] Connection header received
0807 03:03:17.364687 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.364693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.364796 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.364900 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:17.366423 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.367510 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.369805 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.370214 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.370347 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.372806 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:17.372811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:17.372814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:17.372844 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:17.375381 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:17.376197 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:17.376200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:17.376202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:17.376320 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:17.376704 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:17.376709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:17.376711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:17.377034 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:17.377170 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.377693 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.377924 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:03:17.378513 17066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.363366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42317 (local address 127.5.122.3:48841)
0807 03:03:17.364354 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.364377 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.364551 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.364964 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.364970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.365693 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:17.366189 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.366207 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.367740 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.367746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.369638 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.369660 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.370449 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.370460 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.370639 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.371370 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:17.371401 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:17.372619 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:17.375508 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:17.375514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:17.375519 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:17.376099 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:17.376406 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:17.376411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:17.376415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:17.376602 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:17.377191 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:17.377197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:17.377387 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.377539 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.378024 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":609,"threads_started":1}
I20260807 03:03:17.386976 16946 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:33743
I20260807 03:03:17.386999 17148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:33743 every 8 connection(s)
I20260807 03:03:17.388114 16946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:17.390103 17153 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42317
I20260807 03:03:17.390355 17153 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:17.390981 17153 heartbeater.cc:511] Master 127.5.122.62:42317 requested a full tablet report, sending...
I20260807 03:03:17.392256 16414 ts_manager.cc:194] Registered new tserver with Master: abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743)
I20260807 03:03:17.393044 16414 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:34223
I20260807 03:03:17.394767 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 16946
I20260807 03:03:17.394943 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:03:17.395552 5608 external_mini_cluster.cc:1428] Setting key adf2d1dea6364a249c0e402a3c5b05ea
I20260807 03:03:17.398185 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 03:03:17 dist-test-slave-fgb3 krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 03:03:17 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:17 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 03:03:17.452591 16565 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:17.442725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.5.122.62:42317)
0807 03:03:17.442959 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.442965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.443146 (+ 181us) server_negotiation.cc:407] Connection header received
0807 03:03:17.443333 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.443337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.443404 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.443489 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:17.444830 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.445627 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.446862 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.447271 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.447381 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.452383 (+ 5002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52630: BlockingRecv error: recv got EOF from 127.0.0.1:52630 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 03:03:17 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:17.468021 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.455730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.5.122.62:42317)
0807 03:03:17.455914 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.455920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.456175 (+ 255us) server_negotiation.cc:407] Connection header received
0807 03:03:17.456316 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.456322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.456397 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.456474 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:17.457514 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.458205 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.459427 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.459814 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.459928 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.463260 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:17.463265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:17.463268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:17.463298 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:17.465632 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:17.466181 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:17.466185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:17.466187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:17.466299 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:17.466634 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:17.466637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:17.466639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:17.466920 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:17.467171 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.467487 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.467713 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:03:17.471598 16414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:03:17.478282 16414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:17.505455 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.489452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:33743 (local address 127.0.0.1:50634)
0807 03:03:17.496721 (+ 7269us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.496776 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.496977 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.498180 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.498187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.498216 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:17.498688 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.498702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.500536 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.500542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.502605 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.502617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.504487 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.504494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.504668 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.504726 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.504829 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.504995 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6985,"mutex_wait_us":116,"thread_start_us":8922,"threads_started":1}
I20260807 03:03:17.505970 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.490015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.5.122.2:44659)
0807 03:03:17.498298 (+ 8283us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.498305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.498358 (+ 53us) server_negotiation.cc:407] Connection header received
0807 03:03:17.498465 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.498472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.498675 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.498838 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:17.499343 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.500551 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.503299 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.504672 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.504789 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.504985 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.505515 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.505686 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8122,"thread_start_us":7990,"threads_started":1}
I20260807 03:03:17.506281 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.489587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50634 (local address 127.5.122.3:33743)
0807 03:03:17.496344 (+ 6757us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.496353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.497168 (+ 815us) server_negotiation.cc:407] Connection header received
0807 03:03:17.497361 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.497370 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.497627 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.497805 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:17.498935 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.500384 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.502752 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.503990 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.504111 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.504356 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.505835 (+ 1479us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.505990 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6565,"thread_start_us":6391,"threads_started":1}
I20260807 03:03:17.505977 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.489509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:44659 (local address 127.0.0.1:47840)
0807 03:03:17.497704 (+ 8195us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.497719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.497825 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.498891 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.498896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.498920 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:17.499210 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.499218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.500707 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.500713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.503137 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.503157 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.505140 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.505146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.505318 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.505351 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.505455 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.505569 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8085,"mutex_wait_us":48,"thread_start_us":6795,"threads_started":1}
I20260807 03:03:17.506047 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.489962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:36523 (local address 127.0.0.1:52568)
0807 03:03:17.498652 (+ 8690us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.498671 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.498791 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.499396 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.499400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.499425 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:17.499777 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.499789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.501506 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.501511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.503519 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.503529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.505256 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.505263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.505432 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.505468 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.505555 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.505686 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8561,"thread_start_us":7479,"threads_started":1}
I20260807 03:03:17.506779 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.490615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.5.122.1:36523)
0807 03:03:17.497281 (+ 6666us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.497290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.498785 (+ 1495us) server_negotiation.cc:407] Connection header received
0807 03:03:17.498993 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.499001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.499218 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.499418 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:17.499946 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.501368 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.503684 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.505036 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.505147 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.505341 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.506177 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.506386 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6517,"thread_start_us":6401,"threads_started":1}
I20260807 03:03:17.509944 17046 tablet_service.cc:1467] Processing CreateTablet for tablet 1a070e65a63e420996e92285a5f8a7bf (DEFAULT_TABLE table=test-table [id=470e84d93b8e4c9eb0a1711543d8842f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:17.510018 16830 tablet_service.cc:1467] Processing CreateTablet for tablet 1a070e65a63e420996e92285a5f8a7bf (DEFAULT_TABLE table=test-table [id=470e84d93b8e4c9eb0a1711543d8842f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:17.510018 16622 tablet_service.cc:1467] Processing CreateTablet for tablet 1a070e65a63e420996e92285a5f8a7bf (DEFAULT_TABLE table=test-table [id=470e84d93b8e4c9eb0a1711543d8842f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:17.511234 17046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a070e65a63e420996e92285a5f8a7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:17.511350 16830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a070e65a63e420996e92285a5f8a7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:17.511348 16622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a070e65a63e420996e92285a5f8a7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:17.522001 17200 tablet_bootstrap.cc:492] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Bootstrap starting.
I20260807 03:03:17.525272 17201 tablet_bootstrap.cc:492] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Bootstrap starting.
I20260807 03:03:17.525602 17200 tablet_bootstrap.cc:654] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:17.526129 17203 tablet_bootstrap.cc:492] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Bootstrap starting.
I20260807 03:03:17.527743 17200 log.cc:824] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:17.530083 17201 tablet_bootstrap.cc:654] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:17.530423 17200 tablet_bootstrap.cc:492] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: No bootstrap required, opened a new log
I20260807 03:03:17.530680 17200 ts_tablet_manager.cc:1397] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260807 03:03:17.531416 17201 log.cc:824] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:17.531646 17203 tablet_bootstrap.cc:654] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:17.533032 17203 log.cc:824] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:17.536047 17203 tablet_bootstrap.cc:492] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: No bootstrap required, opened a new log
I20260807 03:03:17.536129 17201 tablet_bootstrap.cc:492] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: No bootstrap required, opened a new log
I20260807 03:03:17.536304 17203 ts_tablet_manager.cc:1397] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 03:03:17.536423 17201 ts_tablet_manager.cc:1397] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 03:03:17.539842 17200 raft_consensus.cc:348] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.540282 17200 raft_consensus.cc:374] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:17.540397 17200 raft_consensus.cc:729] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Initialized, Role: FOLLOWER
I20260807 03:03:17.540983 17200 consensus_queue.cc:260] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.542135 17153 heartbeater.cc:503] Master 127.5.122.62:42317 was elected leader, sending a full tablet report...
I20260807 03:03:17.542742 17200 ts_tablet_manager.cc:1428] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 03:03:17.544878 17201 raft_consensus.cc:348] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.545320 17201 raft_consensus.cc:374] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:17.545431 17201 raft_consensus.cc:729] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dd9008e4be471389f553e7a04edb7b, State: Initialized, Role: FOLLOWER
I20260807 03:03:17.545913 17203 raft_consensus.cc:348] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.546286 17203 raft_consensus.cc:374] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:17.546130 17201 consensus_queue.cc:260] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.546403 17203 raft_consensus.cc:729] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c9ed81b70b4905876c97d5e99c5828, State: Initialized, Role: FOLLOWER
I20260807 03:03:17.547274 16937 heartbeater.cc:503] Master 127.5.122.62:42317 was elected leader, sending a full tablet report...
I20260807 03:03:17.547073 17203 consensus_queue.cc:260] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.548017 17201 ts_tablet_manager.cc:1428] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Time spent starting tablet: real 0.011s user 0.013s sys 0.000s
I20260807 03:03:17.548786 17203 ts_tablet_manager.cc:1428] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Time spent starting tablet: real 0.012s user 0.004s sys 0.010s
W20260807 03:03:17.584012 16732 tablet.cc:2369] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:03:17.609997 16938 tablet.cc:2369] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:03:17.639590 17154 tablet.cc:2369] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:03:17.692538 17207 raft_consensus.cc:482] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:17.692836 17207 raft_consensus.cc:504] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.694553 17207 leader_election.cc:288] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659), c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523)
I20260807 03:03:17.700839 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.694952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:54663 (local address 127.5.122.2:44659)
0807 03:03:17.695305 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.695311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.695330 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:03:17.695418 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.695424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.695505 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.695587 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:17.696026 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.697051 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.699021 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.700091 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.700187 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.700265 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.700565 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.700653 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260807 03:03:17.700979 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.694933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:44659 (local address 127.5.122.3:54663)
0807 03:03:17.695161 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.695185 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.695301 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.695636 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.695641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.695671 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:17.695867 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.695882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.697178 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.697184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.698861 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.698880 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.700235 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.700241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.700422 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.700457 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.700551 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.700682 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":16}
I20260807 03:03:17.701773 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.695135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:56977 (local address 127.5.122.1:36523)
0807 03:03:17.695345 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.695352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.695954 (+ 602us) server_negotiation.cc:407] Connection header received
0807 03:03:17.696080 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.696083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.696156 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.696252 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:17.696821 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.697796 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.699860 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.700859 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.700984 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.701079 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.701483 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.701613 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:03:17.701859 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.694965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:36523 (local address 127.5.122.3:56977)
0807 03:03:17.695828 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.695846 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.695978 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.696275 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.696280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.696305 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:17.696661 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.696672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.697926 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.697930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.699699 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.699714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.701207 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.701213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.701330 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.701352 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.701432 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.701511 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":1,"thread_start_us":539,"threads_started":1}
I20260807 03:03:17.702128 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a070e65a63e420996e92285a5f8a7bf" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b" is_pre_election: true
I20260807 03:03:17.702677 16868 raft_consensus.cc:2454] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 0.
I20260807 03:03:17.702805 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a070e65a63e420996e92285a5f8a7bf" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828" is_pre_election: true
I20260807 03:03:17.703271 16652 raft_consensus.cc:2454] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 0.
I20260807 03:03:17.703363 16973 leader_election.cc:302] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, abe5e680a7a24dc9a03f4c8e7c26c1f1; no voters:
I20260807 03:03:17.703724 17207 raft_consensus.cc:2781] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:17.703857 17207 raft_consensus.cc:482] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:17.703959 17207 raft_consensus.cc:3037] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:17.706182 17207 raft_consensus.cc:504] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.706789 17207 leader_election.cc:288] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 election: Requested vote from peers 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659), c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523)
I20260807 03:03:17.707162 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a070e65a63e420996e92285a5f8a7bf" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b"
I20260807 03:03:17.707208 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a070e65a63e420996e92285a5f8a7bf" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828"
I20260807 03:03:17.707329 16868 raft_consensus.cc:3037] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:17.707401 16652 raft_consensus.cc:3037] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:17.709543 16868 raft_consensus.cc:2454] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 1.
I20260807 03:03:17.709597 16652 raft_consensus.cc:2454] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 1.
I20260807 03:03:17.709969 16973 leader_election.cc:302] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, abe5e680a7a24dc9a03f4c8e7c26c1f1; no voters:
I20260807 03:03:17.710232 17207 raft_consensus.cc:2781] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:17.710660 17207 raft_consensus.cc:686] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 LEADER]: Becoming Leader. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Running, Role: LEADER
I20260807 03:03:17.711261 17207 consensus_queue.cc:237] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } }
I20260807 03:03:17.715415 16414 catalog_manager.cc:5495] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } health_report { overall_health: UNKNOWN } } }
I20260807 03:03:17.741279 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.735827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.5.122.3:33743)
0807 03:03:17.736006 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.736012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.736033 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:03:17.736248 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.736252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.736324 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.736455 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:17.736809 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.737488 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.739315 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.739726 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.739816 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.740261 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:17.740740 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:17.740847 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.740988 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.741052 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:03:17.748802 16868 raft_consensus.cc:1261] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Refusing update from remote peer abe5e680a7a24dc9a03f4c8e7c26c1f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:17.748844 16652 raft_consensus.cc:1261] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Refusing update from remote peer abe5e680a7a24dc9a03f4c8e7c26c1f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:17.749706 17207 consensus_queue.cc:1035] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:17.749987 17212 consensus_queue.cc:1035] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:17.761003 17216 mvcc.cc:205] Tried to move back new op lower bound from 7315750083569876992 to 7315750083428933632. Current Snapshot: MvccSnapshot[applied={T|T < 7315750083569876992}]
I20260807 03:03:17.764510 17217 mvcc.cc:205] Tried to move back new op lower bound from 7315750083569876992 to 7315750083428933632. Current Snapshot: MvccSnapshot[applied={T|T < 7315750083569876992}]
I20260807 03:03:17.774008 16414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:17.774219 16414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:17.777360 16414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Sending DeleteTablet for 3 replicas of tablet 1a070e65a63e420996e92285a5f8a7bf
I20260807 03:03:17.778532 17046 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a070e65a63e420996e92285a5f8a7bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:17 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:50634
I20260807 03:03:17.778828 16622 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a070e65a63e420996e92285a5f8a7bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:17 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:52568
I20260807 03:03:17.779230 16830 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a070e65a63e420996e92285a5f8a7bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:17 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47840
I20260807 03:03:17.779788 16414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:03:17.780440 16414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:17.781885 17236 tablet_replica.cc:330] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: stopping tablet replica
I20260807 03:03:17.782287 17236 raft_consensus.cc:2229] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:17.782810 17236 raft_consensus.cc:2258] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:17.783593 17234 tablet_replica.cc:330] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: stopping tablet replica
I20260807 03:03:17.783974 17234 raft_consensus.cc:2229] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:17.784415 17234 raft_consensus.cc:2258] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:17.786484 17236 ts_tablet_manager.cc:1905] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:17.788297 17045 tablet_service.cc:1467] Processing CreateTablet for tablet db11777bf685401ab39e3fb380322f0c (DEFAULT_TABLE table=test-table [id=312bf85b934845adafaa0186c285b3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:17.788981 17045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db11777bf685401ab39e3fb380322f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:17.791270 17233 tablet_replica.cc:330] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: stopping tablet replica
I20260807 03:03:17.791488 16830 tablet_service.cc:1467] Processing CreateTablet for tablet db11777bf685401ab39e3fb380322f0c (DEFAULT_TABLE table=test-table [id=312bf85b934845adafaa0186c285b3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:17.791633 17233 raft_consensus.cc:2229] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:03:17.791911 16830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db11777bf685401ab39e3fb380322f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:17.794471 16622 tablet_service.cc:1467] Processing CreateTablet for tablet db11777bf685401ab39e3fb380322f0c (DEFAULT_TABLE table=test-table [id=312bf85b934845adafaa0186c285b3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:17.794948 16622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db11777bf685401ab39e3fb380322f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:17.797607 17234 ts_tablet_manager.cc:1905] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:17.800683 17201 tablet_bootstrap.cc:492] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: Bootstrap starting.
I20260807 03:03:17.802273 17201 tablet_bootstrap.cc:654] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:17.792344 17233 raft_consensus.cc:2258] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:17.804049 17203 tablet_bootstrap.cc:492] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: Bootstrap starting.
I20260807 03:03:17.805510 17203 tablet_bootstrap.cc:654] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:17.806097 17201 tablet_bootstrap.cc:492] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: No bootstrap required, opened a new log
I20260807 03:03:17.806244 17201 ts_tablet_manager.cc:1397] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260807 03:03:17.806387 17236 ts_tablet_manager.cc:1918] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:17.806458 17233 ts_tablet_manager.cc:1905] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:17.807060 17201 raft_consensus.cc:348] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.807359 17201 raft_consensus.cc:374] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:17.807472 17201 raft_consensus.cc:729] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dd9008e4be471389f553e7a04edb7b, State: Initialized, Role: FOLLOWER
I20260807 03:03:17.807765 17201 consensus_queue.cc:260] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.808326 17201 ts_tablet_manager.cc:1428] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 03:03:17.808604 17236 log.cc:1196] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/1a070e65a63e420996e92285a5f8a7bf
I20260807 03:03:17.809186 17236 ts_tablet_manager.cc:1939] T 1a070e65a63e420996e92285a5f8a7bf P 06dd9008e4be471389f553e7a04edb7b: Deleting consensus metadata
I20260807 03:03:17.810940 16395 catalog_manager.cc:4841] TS 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659): tablet 1a070e65a63e420996e92285a5f8a7bf (table test-table [id=470e84d93b8e4c9eb0a1711543d8842f]) successfully deleted
I20260807 03:03:17.811695 17234 ts_tablet_manager.cc:1918] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:17.811836 17234 log.cc:1196] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/1a070e65a63e420996e92285a5f8a7bf
I20260807 03:03:17.812250 17234 ts_tablet_manager.cc:1939] T 1a070e65a63e420996e92285a5f8a7bf P c0c9ed81b70b4905876c97d5e99c5828: Deleting consensus metadata
I20260807 03:03:17.813586 16393 catalog_manager.cc:4841] TS c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523): tablet 1a070e65a63e420996e92285a5f8a7bf (table test-table [id=470e84d93b8e4c9eb0a1711543d8842f]) successfully deleted
I20260807 03:03:17.814524 17233 ts_tablet_manager.cc:1918] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:17.814692 17233 log.cc:1196] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/1a070e65a63e420996e92285a5f8a7bf
I20260807 03:03:17.815157 17233 ts_tablet_manager.cc:1939] T 1a070e65a63e420996e92285a5f8a7bf P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting consensus metadata
I20260807 03:03:17.816742 16397 catalog_manager.cc:4841] TS abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743): tablet 1a070e65a63e420996e92285a5f8a7bf (table test-table [id=470e84d93b8e4c9eb0a1711543d8842f]) successfully deleted
I20260807 03:03:17.819015 17200 tablet_bootstrap.cc:492] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: Bootstrap starting.
I20260807 03:03:17.820856 17200 tablet_bootstrap.cc:654] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:17.823354 17203 tablet_bootstrap.cc:492] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: No bootstrap required, opened a new log
I20260807 03:03:17.823501 17203 ts_tablet_manager.cc:1397] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.008s
I20260807 03:03:17.824275 17203 raft_consensus.cc:348] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.824483 17203 raft_consensus.cc:374] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:17.824577 17203 raft_consensus.cc:729] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c9ed81b70b4905876c97d5e99c5828, State: Initialized, Role: FOLLOWER
I20260807 03:03:17.824843 17203 consensus_queue.cc:260] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.825348 17203 ts_tablet_manager.cc:1428] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 03:03:17.827080 17200 tablet_bootstrap.cc:492] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: No bootstrap required, opened a new log
I20260807 03:03:17.827250 17200 ts_tablet_manager.cc:1397] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20260807 03:03:17.828135 17200 raft_consensus.cc:348] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.828366 17200 raft_consensus.cc:374] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:17.828469 17200 raft_consensus.cc:729] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Initialized, Role: FOLLOWER
I20260807 03:03:17.828792 17200 consensus_queue.cc:260] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.829895 17200 ts_tablet_manager.cc:1428] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 03:03:17.917490 17210 raft_consensus.cc:482] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:17.917722 17210 raft_consensus.cc:504] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.919618 17210 leader_election.cc:288] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659), abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743)
I20260807 03:03:17.939198 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.920575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:53669 (local address 127.5.122.2:44659)
0807 03:03:17.920760 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.920767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.930658 (+ 9891us) server_negotiation.cc:407] Connection header received
0807 03:03:17.931163 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.931170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.931261 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.931324 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:17.932233 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.933334 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.936485 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.937760 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.937870 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.938005 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.938679 (+ 674us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.938828 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:03:17.940315 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.920517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:56989 (local address 127.5.122.3:33743)
0807 03:03:17.920690 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:17.920696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:17.932763 (+ 12067us) server_negotiation.cc:407] Connection header received
0807 03:03:17.933133 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:17.933139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:17.933218 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:17.933282 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:17.934173 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.935349 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.937787 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.939142 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.939236 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.939326 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:17.939883 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 03:03:17.940020 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:03:17.940913 17242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.920033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:44659 (local address 127.5.122.1:53669)
0807 03:03:17.930541 (+ 10508us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.930567 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.930986 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.931536 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.931541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.931569 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:17.932049 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.932062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.933535 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.933541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.936252 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.936275 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.938227 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.938233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.938463 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.938502 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.940038 (+ 1536us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.940195 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10360,"mutex_wait_us":44,"spinlock_wait_cycles":25472,"thread_start_us":10621,"threads_started":1}
I20260807 03:03:17.941078 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:17.920102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:33743 (local address 127.5.122.1:56989)
0807 03:03:17.932629 (+ 12527us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:17.932647 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:17.932976 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:17.933483 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:17.933488 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:17.933512 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:17.933968 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.933980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.935541 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.935546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.937595 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:17.937610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:17.939523 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:17.939530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:17.939705 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:17.939736 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 03:03:17.940558 (+ 822us) client_negotiation.cc:240] Negotiation successful
0807 03:03:17.940690 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12411,"mutex_wait_us":3,"thread_start_us":12662,"threads_started":1}
I20260807 03:03:17.941982 17081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db11777bf685401ab39e3fb380322f0c" candidate_uuid: "c0c9ed81b70b4905876c97d5e99c5828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" is_pre_election: true
I20260807 03:03:17.942116 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db11777bf685401ab39e3fb380322f0c" candidate_uuid: "c0c9ed81b70b4905876c97d5e99c5828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b" is_pre_election: true
I20260807 03:03:17.942427 16868 raft_consensus.cc:2454] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0c9ed81b70b4905876c97d5e99c5828 in term 0.
I20260807 03:03:17.942407 17081 raft_consensus.cc:2454] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0c9ed81b70b4905876c97d5e99c5828 in term 0.
I20260807 03:03:17.943008 16560 leader_election.cc:302] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe5e680a7a24dc9a03f4c8e7c26c1f1, c0c9ed81b70b4905876c97d5e99c5828; no voters:
I20260807 03:03:17.943475 17210 raft_consensus.cc:2781] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:17.943621 17210 raft_consensus.cc:482] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:17.943717 17210 raft_consensus.cc:3037] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:17.947065 17210 raft_consensus.cc:504] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.948016 17210 leader_election.cc:288] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [CANDIDATE]: Term 1 election: Requested vote from peers 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659), abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743)
I20260807 03:03:17.948370 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db11777bf685401ab39e3fb380322f0c" candidate_uuid: "c0c9ed81b70b4905876c97d5e99c5828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b"
I20260807 03:03:17.948604 16868 raft_consensus.cc:3037] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:17.948941 17081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db11777bf685401ab39e3fb380322f0c" candidate_uuid: "c0c9ed81b70b4905876c97d5e99c5828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1"
I20260807 03:03:17.949163 17081 raft_consensus.cc:3037] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:17.951550 16868 raft_consensus.cc:2454] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0c9ed81b70b4905876c97d5e99c5828 in term 1.
I20260807 03:03:17.951969 16559 leader_election.cc:302] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, c0c9ed81b70b4905876c97d5e99c5828; no voters:
I20260807 03:03:17.952316 17210 raft_consensus.cc:2781] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:17.952540 17081 raft_consensus.cc:2454] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0c9ed81b70b4905876c97d5e99c5828 in term 1.
I20260807 03:03:17.953416 17210 raft_consensus.cc:686] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 1 LEADER]: Becoming Leader. State: Replica: c0c9ed81b70b4905876c97d5e99c5828, State: Running, Role: LEADER
I20260807 03:03:17.954046 17210 consensus_queue.cc:237] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:17.957837 16414 catalog_manager.cc:5495] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1). New cstate: current_term: 1 leader_uuid: "c0c9ed81b70b4905876c97d5e99c5828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
Aug 07 03:03:18 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071795, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.62@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:18.071563 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.033337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42317 (local address 127.0.0.1:52648)
0807 03:03:18.048789 (+ 15452us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:18.048818 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:18.049008 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:18.049443 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:18.049449 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:18.050411 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:18.050675 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.050688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.052188 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.052194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.053691 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.053706 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.054774 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.054780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.054971 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.061222 (+ 6251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:18.061259 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:18.064427 (+ 3168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:18.068371 (+ 3944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:18.068386 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:18.068394 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:18.069008 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:18.069443 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:18.069448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:18.069452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:18.069653 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:18.070303 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:18.070347 (+ 44us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:18.070572 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 03:03:18.070742 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 03:03:18.071067 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14909,"thread_start_us":15057,"threads_started":1}
I20260807 03:03:18.072042 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.033466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52648 (local address 127.5.122.62:42317)
0807 03:03:18.049056 (+ 15590us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.049063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.049089 (+ 26us) server_negotiation.cc:407] Connection header received
0807 03:03:18.049155 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.049160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.049243 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.049374 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:18.050843 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.052030 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.053852 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.054592 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.054734 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.064694 (+ 9960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:18.064702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:18.064707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:18.064748 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:18.068218 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:18.069153 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:18.069159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:18.069162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:18.069322 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:18.069782 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:18.069787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:18.069791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:18.070164 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:18.070341 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.070896 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.071301 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":15434,"thread_start_us":15645,"threads_started":1}
I20260807 03:03:18.078249 16414 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:52648:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 03:03:18.079634 16414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:18.087927 16622 tablet_service.cc:1467] Processing CreateTablet for tablet f34dd402d9264233b1c867e1e2635a1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eece57b16b244e08b7de17db30337f43]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:03:18.088333 16830 tablet_service.cc:1467] Processing CreateTablet for tablet f34dd402d9264233b1c867e1e2635a1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eece57b16b244e08b7de17db30337f43]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:03:18.088564 16622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34dd402d9264233b1c867e1e2635a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:18.088904 16830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34dd402d9264233b1c867e1e2635a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:18.092079 17045 tablet_service.cc:1467] Processing CreateTablet for tablet f34dd402d9264233b1c867e1e2635a1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eece57b16b244e08b7de17db30337f43]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 03:03:18.092618 17045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34dd402d9264233b1c867e1e2635a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:18.097613 17203 tablet_bootstrap.cc:492] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: Bootstrap starting.
I20260807 03:03:18.099416 17203 tablet_bootstrap.cc:654] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:18.104151 17201 tablet_bootstrap.cc:492] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: Bootstrap starting.
I20260807 03:03:18.105890 17201 tablet_bootstrap.cc:654] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:18.106992 17200 tablet_bootstrap.cc:492] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: Bootstrap starting.
I20260807 03:03:18.108906 17200 tablet_bootstrap.cc:654] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:18.120409 17201 tablet_bootstrap.cc:492] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: No bootstrap required, opened a new log
I20260807 03:03:18.120600 17201 ts_tablet_manager.cc:1397] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20260807 03:03:18.121398 17201 raft_consensus.cc:348] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.121658 17201 raft_consensus.cc:374] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:18.121733 17201 raft_consensus.cc:729] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dd9008e4be471389f553e7a04edb7b, State: Initialized, Role: FOLLOWER
I20260807 03:03:18.122021 17201 consensus_queue.cc:260] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.122489 17209 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.122829 17209 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:03:18.123951 17201 ts_tablet_manager.cc:1428] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 03:03:18.126503 17200 tablet_bootstrap.cc:492] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: No bootstrap required, opened a new log
I20260807 03:03:18.126677 17200 ts_tablet_manager.cc:1397] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.002s
I20260807 03:03:18.127324 17200 raft_consensus.cc:348] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.127516 17200 raft_consensus.cc:374] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:18.127599 17200 raft_consensus.cc:729] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Initialized, Role: FOLLOWER
I20260807 03:03:18.127857 17200 consensus_queue.cc:260] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.128983 17212 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.129317 17212 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:03:18.130935 17200 ts_tablet_manager.cc:1428] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 03:03:18.133477 17209 raft_consensus.cc:482] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:18.133663 17209 raft_consensus.cc:504] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.134029 17203 tablet_bootstrap.cc:492] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: No bootstrap required, opened a new log
I20260807 03:03:18.134207 17203 ts_tablet_manager.cc:1397] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.003s
I20260807 03:03:18.135025 17203 raft_consensus.cc:348] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.135499 17209 leader_election.cc:288] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523), abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743)
I20260807 03:03:18.135277 17203 raft_consensus.cc:374] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:18.141465 17203 raft_consensus.cc:729] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c9ed81b70b4905876c97d5e99c5828, State: Initialized, Role: FOLLOWER
I20260807 03:03:18.142045 17203 consensus_queue.cc:260] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.143024 17210 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.143324 17210 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:03:18.146175 17203 ts_tablet_manager.cc:1428] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260807 03:03:18.154879 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.136144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:52897 (local address 127.5.122.1:36523)
0807 03:03:18.136319 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.136325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.147154 (+ 10829us) server_negotiation.cc:407] Connection header received
0807 03:03:18.147451 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.147457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.147555 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.147615 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:18.148400 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.149537 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.152331 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.153589 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.153680 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.153781 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.154459 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.154598 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:03:18.155633 17261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.135696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:36523 (local address 127.5.122.2:52897)
0807 03:03:18.147055 (+ 11359us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:18.147076 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:18.147290 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:18.147821 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:18.147826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:18.147855 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:18.148207 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.148221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.149724 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.149731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.152127 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.152144 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.153975 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.153982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.154235 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.154283 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 03:03:18.155151 (+ 868us) client_negotiation.cc:240] Negotiation successful
0807 03:03:18.155302 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9915,"thread_start_us":17842,"threads_started":1}
I20260807 03:03:18.156673 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34dd402d9264233b1c867e1e2635a1f" candidate_uuid: "06dd9008e4be471389f553e7a04edb7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828" is_pre_election: true
I20260807 03:03:18.156906 16652 raft_consensus.cc:2454] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06dd9008e4be471389f553e7a04edb7b in term 0.
I20260807 03:03:18.157644 16754 leader_election.cc:302] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, c0c9ed81b70b4905876c97d5e99c5828; no voters:
I20260807 03:03:18.158035 17209 raft_consensus.cc:2781] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:18.158172 17209 raft_consensus.cc:482] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:18.158305 17209 raft_consensus.cc:3037] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:18.161494 17209 raft_consensus.cc:504] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.162685 17209 leader_election.cc:288] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [CANDIDATE]: Term 1 election: Requested vote from peers c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523), abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743)
I20260807 03:03:18.162775 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34dd402d9264233b1c867e1e2635a1f" candidate_uuid: "06dd9008e4be471389f553e7a04edb7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828"
I20260807 03:03:18.162973 16652 raft_consensus.cc:3037] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:18.164870 16652 raft_consensus.cc:2454] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06dd9008e4be471389f553e7a04edb7b in term 1.
I20260807 03:03:18.165228 16754 leader_election.cc:302] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, c0c9ed81b70b4905876c97d5e99c5828; no voters:
I20260807 03:03:18.165552 17209 raft_consensus.cc:2781] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:18.165865 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.136541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:49891 (local address 127.5.122.3:33743)
0807 03:03:18.136710 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.136716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.153517 (+ 16801us) server_negotiation.cc:407] Connection header received
0807 03:03:18.154078 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.154084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.154184 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.154248 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:18.155124 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.156112 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.160584 (+ 4472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.161949 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.162051 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.162160 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.165381 (+ 3221us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.165538 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:03:18.166641 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.136056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:33743 (local address 127.5.122.2:49891)
0807 03:03:18.153413 (+ 17357us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:18.153433 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:18.153870 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:18.154498 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:18.154502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:18.154527 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:18.154861 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.154874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.158073 (+ 3199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.158078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.160399 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.160414 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.162428 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.162434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.165184 (+ 2750us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.165227 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 03:03:18.166107 (+ 880us) client_negotiation.cc:240] Negotiation successful
0807 03:03:18.166266 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17230,"thread_start_us":10725,"threads_started":1}
I20260807 03:03:18.167444 17081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34dd402d9264233b1c867e1e2635a1f" candidate_uuid: "06dd9008e4be471389f553e7a04edb7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" is_pre_election: true
I20260807 03:03:18.167500 17080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34dd402d9264233b1c867e1e2635a1f" candidate_uuid: "06dd9008e4be471389f553e7a04edb7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1"
I20260807 03:03:18.167743 17081 raft_consensus.cc:2454] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06dd9008e4be471389f553e7a04edb7b in term 0.
I20260807 03:03:18.169090 17209 raft_consensus.cc:686] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [term 1 LEADER]: Becoming Leader. State: Replica: 06dd9008e4be471389f553e7a04edb7b, State: Running, Role: LEADER
I20260807 03:03:18.169736 17209 consensus_queue.cc:237] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.171859 17262 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: TxnStatusTablet state changed. Reason: New leader 06dd9008e4be471389f553e7a04edb7b. Latest consensus state: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.172487 17262 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:03:18.173592 16414 catalog_manager.cc:5495] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 06dd9008e4be471389f553e7a04edb7b (127.5.122.2). New cstate: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } health_report { overall_health: UNKNOWN } } }
I20260807 03:03:18.178332 17266 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 03:03:18.178570 17266 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 03:03:18.214743 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.206957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:44659 (local address 127.0.0.1:47844)
0807 03:03:18.207207 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:18.207229 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:18.207387 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:18.208197 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:18.208201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:18.208210 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:03:18.208384 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.208395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.209880 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.209886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.211515 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.211532 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.212630 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.212637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.212871 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.212946 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:03:18.214113 (+ 1167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:03:18.214146 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:03:18.214227 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 03:03:18.214412 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 03:03:18.215461 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.207505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.5.122.2:44659)
0807 03:03:18.207701 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.207708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.207726 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:03:18.207817 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.207821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.207905 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.208046 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:18.208521 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.209694 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.211694 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.212352 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.212485 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.213177 (+ 692us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:18.213902 (+ 725us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:18.214049 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.215068 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.215156 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 03:03:18.219923 16652 raft_consensus.cc:1261] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Refusing update from remote peer 06dd9008e4be471389f553e7a04edb7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:18.219957 17080 raft_consensus.cc:3037] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:18.220636 17262 consensus_queue.cc:1035] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:18.223474 17080 raft_consensus.cc:1261] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Refusing update from remote peer 06dd9008e4be471389f553e7a04edb7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:18.224177 17262 consensus_queue.cc:1035] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:18.241112 17210 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: TxnStatusTablet state changed. Reason: New leader 06dd9008e4be471389f553e7a04edb7b. Latest consensus state: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.241477 17210 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:03:18.243204 17212 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: TxnStatusTablet state changed. Reason: New leader 06dd9008e4be471389f553e7a04edb7b. Latest consensus state: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.243534 17212 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:03:18.247458 17262 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.247697 17262 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:03:18.248205 17210 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.248502 17210 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P c0c9ed81b70b4905876c97d5e99c5828: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:03:18.284994 17273 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.285724 17273 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P 06dd9008e4be471389f553e7a04edb7b: This TxnStatusTablet replica's current role is: LEADER
I20260807 03:03:18.287252 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.279378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.5.122.1:36523)
0807 03:03:18.279562 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.279568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.279677 (+ 109us) server_negotiation.cc:407] Connection header received
0807 03:03:18.280068 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.280075 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.280204 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.280336 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:18.281065 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.282221 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.284276 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.284904 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.285012 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.285626 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:18.286455 (+ 829us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:18.286574 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.286814 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.286922 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:03:18.328817 17212 tablet_replica.cc:439] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06dd9008e4be471389f553e7a04edb7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } } }
I20260807 03:03:18.329137 17212 tablet_replica.cc:442] T f34dd402d9264233b1c867e1e2635a1f P abe5e680a7a24dc9a03f4c8e7c26c1f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 03:03:18.332086 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.314645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:59573 (local address 127.5.122.2:44659)
0807 03:03:18.314843 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.314849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.323618 (+ 8769us) server_negotiation.cc:407] Connection header received
0807 03:03:18.324129 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.324136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.324240 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.324303 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:18.325029 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.326251 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.329607 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.330350 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.330448 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.331278 (+ 830us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:18.331409 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:18.331471 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.331698 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.331782 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:03:18.332866 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.314210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:44659 (local address 127.5.122.1:59573)
0807 03:03:18.323499 (+ 9289us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:18.323521 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:18.323946 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:18.324496 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:18.324501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:18.324515 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:03:18.324848 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.324858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.326472 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.326477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.328166 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.328181 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.330772 (+ 2591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.330778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.330960 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.331036 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:03:18.331561 (+ 525us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:03:18.331573 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 03:03:18.332327 (+ 754us) client_negotiation.cc:240] Negotiation successful
0807 03:03:18.332472 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9117,"thread_start_us":9427,"threads_started":1}
I20260807 03:03:18.342108 17080 raft_consensus.cc:1261] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Refusing update from remote peer c0c9ed81b70b4905876c97d5e99c5828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:18.342121 16868 raft_consensus.cc:1261] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Refusing update from remote peer c0c9ed81b70b4905876c97d5e99c5828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:18.342834 17243 consensus_queue.cc:1035] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:18.343439 17243 consensus_queue.cc:1035] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:18.352532 17218 mvcc.cc:205] Tried to move back new op lower bound from 7315750086004088832 to 7315750084423454720. Current Snapshot: MvccSnapshot[applied={T|T < 7315750086004088832}]
I20260807 03:03:18.417824 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.388685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:47043 (local address 127.5.122.1:36523)
0807 03:03:18.388852 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.388858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.407117 (+ 18259us) server_negotiation.cc:407] Connection header received
0807 03:03:18.407374 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.407390 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.407470 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.407525 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:18.410185 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.411356 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.415577 (+ 4221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.416277 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.416363 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.417057 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:18.417201 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:18.417316 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.417432 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.417519 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:03:18.417969 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.388302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:36523 (local address 127.5.122.2:47043)
0807 03:03:18.407019 (+ 18717us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:18.407040 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:18.407234 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:18.409573 (+ 2339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:18.409578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:18.409595 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:03:18.409978 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.409990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.411867 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.411872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.415415 (+ 3543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:18.415431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.416583 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.416590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:18.416756 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.416831 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:03:18.417313 (+ 482us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:03:18.417324 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 03:03:18.417403 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 03:03:18.417546 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18416,"thread_start_us":18264,"threads_started":1}
I20260807 03:03:18.464525 16414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:18.464843 16414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:18.468452 16414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Sending DeleteTablet for 3 replicas of tablet db11777bf685401ab39e3fb380322f0c
I20260807 03:03:18.469280 16622 tablet_service.cc:1514] Processing DeleteTablet for tablet db11777bf685401ab39e3fb380322f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:18 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:52568
I20260807 03:03:18.469779 17045 tablet_service.cc:1514] Processing DeleteTablet for tablet db11777bf685401ab39e3fb380322f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:18 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:50634
I20260807 03:03:18.470368 16830 tablet_service.cc:1514] Processing DeleteTablet for tablet db11777bf685401ab39e3fb380322f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:18 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47840
I20260807 03:03:18.493765 17300 tablet_replica.cc:330] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: stopping tablet replica
I20260807 03:03:18.494356 17301 tablet_replica.cc:330] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: stopping tablet replica
I20260807 03:03:18.494726 17300 raft_consensus.cc:2229] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:18.495034 17301 raft_consensus.cc:2229] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:18.495539 17300 raft_consensus.cc:2258] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:18.495802 17301 raft_consensus.cc:2258] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:18.497548 17301 ts_tablet_manager.cc:1905] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:18.498128 17300 ts_tablet_manager.cc:1905] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:18.506816 17298 tablet_replica.cc:330] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: stopping tablet replica
I20260807 03:03:18.507109 17298 raft_consensus.cc:2229] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:03:18.507912 17298 raft_consensus.cc:2258] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:18.509189 17298 ts_tablet_manager.cc:1905] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:18.516439 17298 ts_tablet_manager.cc:1918] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:03:18.516649 17298 log.cc:1196] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/db11777bf685401ab39e3fb380322f0c
I20260807 03:03:18.517290 17300 ts_tablet_manager.cc:1918] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:03:18.517467 17300 log.cc:1196] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/db11777bf685401ab39e3fb380322f0c
I20260807 03:03:18.517736 17298 ts_tablet_manager.cc:1939] T db11777bf685401ab39e3fb380322f0c P c0c9ed81b70b4905876c97d5e99c5828: Deleting consensus metadata
I20260807 03:03:18.517939 17300 ts_tablet_manager.cc:1939] T db11777bf685401ab39e3fb380322f0c P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting consensus metadata
I20260807 03:03:18.519141 16393 catalog_manager.cc:4841] TS c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523): tablet db11777bf685401ab39e3fb380322f0c (table test-table [id=312bf85b934845adafaa0186c285b3f8]) successfully deleted
I20260807 03:03:18.519297 16397 catalog_manager.cc:4841] TS abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743): tablet db11777bf685401ab39e3fb380322f0c (table test-table [id=312bf85b934845adafaa0186c285b3f8]) successfully deleted
I20260807 03:03:18.523065 17301 ts_tablet_manager.cc:1918] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:03:18.523306 17301 log.cc:1196] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/db11777bf685401ab39e3fb380322f0c
I20260807 03:03:18.523859 17301 ts_tablet_manager.cc:1939] T db11777bf685401ab39e3fb380322f0c P 06dd9008e4be471389f553e7a04edb7b: Deleting consensus metadata
I20260807 03:03:18.525101 16395 catalog_manager.cc:4841] TS 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659): tablet db11777bf685401ab39e3fb380322f0c (table test-table [id=312bf85b934845adafaa0186c285b3f8]) successfully deleted
Aug 07 03:03:18 dist-test-slave-fgb3 krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 03:03:18 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:18 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 03:03:18.573554 17254 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:18.560737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52660 (local address 127.5.122.62:42317)
0807 03:03:18.560994 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.561000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.561344 (+ 344us) server_negotiation.cc:407] Connection header received
0807 03:03:18.561541 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.561548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.561633 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.561744 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:18.563482 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.564621 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.566438 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.567090 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.567217 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.573295 (+ 6078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52660: BlockingRecv error: recv got EOF from 127.0.0.1:52660 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 03:03:18 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM
I20260807 03:03:18.595036 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:18.577246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.5.122.62:42317)
0807 03:03:18.577410 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:18.577415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:18.577720 (+ 305us) server_negotiation.cc:407] Connection header received
0807 03:03:18.577922 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:18.577928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:18.578018 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:18.578116 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:18.579733 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.580850 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.582777 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:18.583442 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:18.583565 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:18.588430 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:18.588437 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:18.588442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:18.588485 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:18.592006 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:18.592852 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:18.592858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:18.592862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:18.593030 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:18.593450 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:18.593456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:18.593459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:18.593842 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:18.594124 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:18.594427 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 03:03:18.594663 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 03:03:18.599375 16414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:03:18.600132 16414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:18.607877 16622 tablet_service.cc:1467] Processing CreateTablet for tablet a531db076fa6401fbff1cb133864d2fa (DEFAULT_TABLE table=test-table [id=e2b8ca95f4114d1e8de4654ee76fc53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:18.608330 16622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a531db076fa6401fbff1cb133864d2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:18.608425 16830 tablet_service.cc:1467] Processing CreateTablet for tablet a531db076fa6401fbff1cb133864d2fa (DEFAULT_TABLE table=test-table [id=e2b8ca95f4114d1e8de4654ee76fc53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:18.608880 16830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a531db076fa6401fbff1cb133864d2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:18.614830 17203 tablet_bootstrap.cc:492] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: Bootstrap starting.
I20260807 03:03:18.614976 17045 tablet_service.cc:1467] Processing CreateTablet for tablet a531db076fa6401fbff1cb133864d2fa (DEFAULT_TABLE table=test-table [id=e2b8ca95f4114d1e8de4654ee76fc53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:18.615479 17045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a531db076fa6401fbff1cb133864d2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:18.616629 17203 tablet_bootstrap.cc:654] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:18.620126 17201 tablet_bootstrap.cc:492] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: Bootstrap starting.
I20260807 03:03:18.621872 17201 tablet_bootstrap.cc:654] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:18.625777 17200 tablet_bootstrap.cc:492] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: Bootstrap starting.
I20260807 03:03:18.627511 17200 tablet_bootstrap.cc:654] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:18.634773 17203 tablet_bootstrap.cc:492] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: No bootstrap required, opened a new log
I20260807 03:03:18.634948 17203 ts_tablet_manager.cc:1397] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.013s
I20260807 03:03:18.634997 17201 tablet_bootstrap.cc:492] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: No bootstrap required, opened a new log
I20260807 03:03:18.635174 17201 ts_tablet_manager.cc:1397] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260807 03:03:18.635782 17203 raft_consensus.cc:348] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.636063 17203 raft_consensus.cc:374] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:18.636163 17203 raft_consensus.cc:729] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c9ed81b70b4905876c97d5e99c5828, State: Initialized, Role: FOLLOWER
I20260807 03:03:18.635983 17201 raft_consensus.cc:348] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.636288 17201 raft_consensus.cc:374] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:18.636492 17201 raft_consensus.cc:729] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dd9008e4be471389f553e7a04edb7b, State: Initialized, Role: FOLLOWER
I20260807 03:03:18.636409 17203 consensus_queue.cc:260] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.636776 17201 consensus_queue.cc:260] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.638336 17201 ts_tablet_manager.cc:1428] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 03:03:18.639853 17203 ts_tablet_manager.cc:1428] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260807 03:03:18.643440 17200 tablet_bootstrap.cc:492] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: No bootstrap required, opened a new log
I20260807 03:03:18.643600 17200 ts_tablet_manager.cc:1397] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260807 03:03:18.644356 17200 raft_consensus.cc:348] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.644608 17200 raft_consensus.cc:374] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:18.644702 17200 raft_consensus.cc:729] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Initialized, Role: FOLLOWER
I20260807 03:03:18.644932 17200 consensus_queue.cc:260] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.645450 17200 ts_tablet_manager.cc:1428] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 03:03:18.963001 17212 raft_consensus.cc:482] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:18.963195 17212 raft_consensus.cc:504] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.963733 17212 leader_election.cc:288] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523), 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659)
I20260807 03:03:18.964246 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a531db076fa6401fbff1cb133864d2fa" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b" is_pre_election: true
I20260807 03:03:18.964241 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a531db076fa6401fbff1cb133864d2fa" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828" is_pre_election: true
I20260807 03:03:18.964473 16652 raft_consensus.cc:2454] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 0.
I20260807 03:03:18.964468 16868 raft_consensus.cc:2454] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 0.
I20260807 03:03:18.964983 16973 leader_election.cc:302] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, abe5e680a7a24dc9a03f4c8e7c26c1f1; no voters:
I20260807 03:03:18.965281 17212 raft_consensus.cc:2781] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:18.965406 17212 raft_consensus.cc:482] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:18.965519 17212 raft_consensus.cc:3037] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:18.967525 17212 raft_consensus.cc:504] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.968014 17212 leader_election.cc:288] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 election: Requested vote from peers c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523), 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659)
I20260807 03:03:18.968432 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a531db076fa6401fbff1cb133864d2fa" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b"
I20260807 03:03:18.968444 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a531db076fa6401fbff1cb133864d2fa" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828"
I20260807 03:03:18.968592 16868 raft_consensus.cc:3037] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:18.968604 16652 raft_consensus.cc:3037] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:18.970479 16868 raft_consensus.cc:2454] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 1.
I20260807 03:03:18.970582 16652 raft_consensus.cc:2454] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 1.
I20260807 03:03:18.970822 16973 leader_election.cc:302] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, abe5e680a7a24dc9a03f4c8e7c26c1f1; no voters:
I20260807 03:03:18.971088 17212 raft_consensus.cc:2781] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:18.972347 17212 raft_consensus.cc:686] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 LEADER]: Becoming Leader. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Running, Role: LEADER
I20260807 03:03:18.972601 17212 consensus_queue.cc:237] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:18.975278 16414 catalog_manager.cc:5495] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260807 03:03:19.072913 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.068394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.5.122.3:33743)
0807 03:03:19.068856 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:19.068862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:19.068879 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:03:19.068940 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:19.068943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:19.069007 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:19.069087 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:19.069390 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.070153 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.071421 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.071849 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.071947 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.072286 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:19.072391 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:19.072477 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:19.072557 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 03:03:19.072617 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":254,"threads_started":1}
I20260807 03:03:19.075801 16652 raft_consensus.cc:1261] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Refusing update from remote peer abe5e680a7a24dc9a03f4c8e7c26c1f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:19.075829 16868 raft_consensus.cc:1261] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Refusing update from remote peer abe5e680a7a24dc9a03f4c8e7c26c1f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:19.076377 17334 consensus_queue.cc:1035] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:19.076695 17212 consensus_queue.cc:1035] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:19.090432 16413 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:19.090631 16413 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:19.093207 16413 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Sending DeleteTablet for 3 replicas of tablet a531db076fa6401fbff1cb133864d2fa
I20260807 03:03:19.093865 16622 tablet_service.cc:1514] Processing DeleteTablet for tablet a531db076fa6401fbff1cb133864d2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:19 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:52568
I20260807 03:03:19.093984 16830 tablet_service.cc:1514] Processing DeleteTablet for tablet a531db076fa6401fbff1cb133864d2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:19 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47840
I20260807 03:03:19.094020 17045 tablet_service.cc:1514] Processing DeleteTablet for tablet a531db076fa6401fbff1cb133864d2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:19 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:50634
I20260807 03:03:19.095670 16413 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:03:19.096385 16413 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:19.102895 17358 tablet_replica.cc:330] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: stopping tablet replica
I20260807 03:03:19.103394 17357 tablet_replica.cc:330] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: stopping tablet replica
I20260807 03:03:19.103649 17357 raft_consensus.cc:2229] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:19.103780 16827 tablet_service.cc:1467] Processing CreateTablet for tablet a53dad9c192246929855c0ae994c07fc (DEFAULT_TABLE table=test-table [id=a9bc4248bced4fd3a152932b5e652f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:19.103930 17358 raft_consensus.cc:2229] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:03:19.103610 16621 tablet_service.cc:1467] Processing CreateTablet for tablet a53dad9c192246929855c0ae994c07fc (DEFAULT_TABLE table=test-table [id=a9bc4248bced4fd3a152932b5e652f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:19.104292 16827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53dad9c192246929855c0ae994c07fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:19.104526 16621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53dad9c192246929855c0ae994c07fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:19.104854 17358 raft_consensus.cc:2258] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:19.105005 17357 raft_consensus.cc:2258] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:19.107147 17357 ts_tablet_manager.cc:1905] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:19.110455 17203 tablet_bootstrap.cc:492] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: Bootstrap starting.
I20260807 03:03:19.111728 17201 tablet_bootstrap.cc:492] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: Bootstrap starting.
I20260807 03:03:19.111879 17359 tablet_replica.cc:330] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: stopping tablet replica
I20260807 03:03:19.112053 17359 raft_consensus.cc:2229] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:19.112176 17203 tablet_bootstrap.cc:654] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:19.112337 17359 raft_consensus.cc:2258] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:19.113375 17201 tablet_bootstrap.cc:654] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:19.114212 17357 ts_tablet_manager.cc:1918] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:19.114418 17357 log.cc:1196] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/a531db076fa6401fbff1cb133864d2fa
I20260807 03:03:19.114457 17203 tablet_bootstrap.cc:492] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: No bootstrap required, opened a new log
I20260807 03:03:19.114681 17203 ts_tablet_manager.cc:1397] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 03:03:19.115038 17357 ts_tablet_manager.cc:1939] T a531db076fa6401fbff1cb133864d2fa P c0c9ed81b70b4905876c97d5e99c5828: Deleting consensus metadata
I20260807 03:03:19.115240 17201 tablet_bootstrap.cc:492] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: No bootstrap required, opened a new log
I20260807 03:03:19.115386 17201 ts_tablet_manager.cc:1397] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 03:03:19.115450 17203 raft_consensus.cc:348] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.115691 17203 raft_consensus.cc:374] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:19.115780 17203 raft_consensus.cc:729] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c9ed81b70b4905876c97d5e99c5828, State: Initialized, Role: FOLLOWER
I20260807 03:03:19.116014 17203 consensus_queue.cc:260] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.116107 16393 catalog_manager.cc:4841] TS c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523): tablet a531db076fa6401fbff1cb133864d2fa (table test-table [id=e2b8ca95f4114d1e8de4654ee76fc53d]) successfully deleted
I20260807 03:03:19.116078 17201 raft_consensus.cc:348] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.116309 17201 raft_consensus.cc:374] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:19.116384 17201 raft_consensus.cc:729] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dd9008e4be471389f553e7a04edb7b, State: Initialized, Role: FOLLOWER
I20260807 03:03:19.116576 17201 consensus_queue.cc:260] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.116956 17203 ts_tablet_manager.cc:1428] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 03:03:19.117075 17201 ts_tablet_manager.cc:1428] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 03:03:19.117625 17045 tablet_service.cc:1467] Processing CreateTablet for tablet a53dad9c192246929855c0ae994c07fc (DEFAULT_TABLE table=test-table [id=a9bc4248bced4fd3a152932b5e652f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:19.118079 17045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53dad9c192246929855c0ae994c07fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:19.118738 17359 ts_tablet_manager.cc:1905] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:19.122917 17358 ts_tablet_manager.cc:1905] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:19.123482 17200 tablet_bootstrap.cc:492] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: Bootstrap starting.
I20260807 03:03:19.124871 17200 tablet_bootstrap.cc:654] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:19.125262 17359 ts_tablet_manager.cc:1918] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:19.125386 17359 log.cc:1196] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/a531db076fa6401fbff1cb133864d2fa
I20260807 03:03:19.125790 17359 ts_tablet_manager.cc:1939] T a531db076fa6401fbff1cb133864d2fa P 06dd9008e4be471389f553e7a04edb7b: Deleting consensus metadata
I20260807 03:03:19.126818 16395 catalog_manager.cc:4841] TS 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659): tablet a531db076fa6401fbff1cb133864d2fa (table test-table [id=e2b8ca95f4114d1e8de4654ee76fc53d]) successfully deleted
I20260807 03:03:19.127596 17200 tablet_bootstrap.cc:492] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: No bootstrap required, opened a new log
I20260807 03:03:19.127756 17200 ts_tablet_manager.cc:1397] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 03:03:19.128381 17200 raft_consensus.cc:348] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.128578 17200 raft_consensus.cc:374] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:19.128639 17358 ts_tablet_manager.cc:1918] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:19.128664 17200 raft_consensus.cc:729] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Initialized, Role: FOLLOWER
I20260807 03:03:19.128793 17358 log.cc:1196] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/a531db076fa6401fbff1cb133864d2fa
I20260807 03:03:19.128968 17200 consensus_queue.cc:260] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.129160 17358 ts_tablet_manager.cc:1939] T a531db076fa6401fbff1cb133864d2fa P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting consensus metadata
I20260807 03:03:19.129393 17200 ts_tablet_manager.cc:1428] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 03:03:19.130153 16397 catalog_manager.cc:4841] TS abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743): tablet a531db076fa6401fbff1cb133864d2fa (table test-table [id=e2b8ca95f4114d1e8de4654ee76fc53d]) successfully deleted
I20260807 03:03:19.188066 17355 raft_consensus.cc:482] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:19.188282 17355 raft_consensus.cc:504] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.189042 17355 leader_election.cc:288] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523), 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659)
I20260807 03:03:19.189618 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53dad9c192246929855c0ae994c07fc" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828" is_pre_election: true
I20260807 03:03:19.189667 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53dad9c192246929855c0ae994c07fc" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b" is_pre_election: true
I20260807 03:03:19.189831 16652 raft_consensus.cc:2454] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 0.
I20260807 03:03:19.189929 16868 raft_consensus.cc:2454] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 0.
I20260807 03:03:19.190263 16970 leader_election.cc:302] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe5e680a7a24dc9a03f4c8e7c26c1f1, c0c9ed81b70b4905876c97d5e99c5828; no voters:
I20260807 03:03:19.190577 17355 raft_consensus.cc:2781] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:19.190680 17355 raft_consensus.cc:482] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:19.190744 17355 raft_consensus.cc:3037] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:19.192890 17355 raft_consensus.cc:504] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.193423 17355 leader_election.cc:288] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 election: Requested vote from peers c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523), 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659)
I20260807 03:03:19.193845 16652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53dad9c192246929855c0ae994c07fc" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c9ed81b70b4905876c97d5e99c5828"
I20260807 03:03:19.193908 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53dad9c192246929855c0ae994c07fc" candidate_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06dd9008e4be471389f553e7a04edb7b"
I20260807 03:03:19.194031 16652 raft_consensus.cc:3037] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:19.194073 16868 raft_consensus.cc:3037] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:19.196419 16868 raft_consensus.cc:2454] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 1.
I20260807 03:03:19.196544 16652 raft_consensus.cc:2454] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe5e680a7a24dc9a03f4c8e7c26c1f1 in term 1.
I20260807 03:03:19.196835 16973 leader_election.cc:302] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dd9008e4be471389f553e7a04edb7b, abe5e680a7a24dc9a03f4c8e7c26c1f1; no voters:
I20260807 03:03:19.197117 17355 raft_consensus.cc:2781] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:19.197256 17355 raft_consensus.cc:686] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 LEADER]: Becoming Leader. State: Replica: abe5e680a7a24dc9a03f4c8e7c26c1f1, State: Running, Role: LEADER
I20260807 03:03:19.197530 17355 consensus_queue.cc:237] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } }
I20260807 03:03:19.199914 16416 catalog_manager.cc:5495] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe5e680a7a24dc9a03f4c8e7c26c1f1" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33743 } health_report { overall_health: HEALTHY } } }
I20260807 03:03:19.239503 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.234652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:39495 (local address 127.5.122.2:44659)
0807 03:03:19.235158 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:19.235165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:19.235339 (+ 174us) server_negotiation.cc:407] Connection header received
0807 03:03:19.235488 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:19.235491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:19.235555 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:19.235631 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:19.236072 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.236760 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.238033 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.238479 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.238568 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.238940 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:19.239036 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:19.239098 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:19.239194 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 03:03:19.239257 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":250,"threads_started":1}
I20260807 03:03:19.239557 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.234655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:44659 (local address 127.5.122.3:39495)
0807 03:03:19.235257 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:19.235272 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:19.235390 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:19.235663 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:19.235667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:19.235676 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:03:19.235943 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:19.235951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.236853 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.236856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:19.237926 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:19.237937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.238613 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.238618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:19.238737 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.238791 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:03:19.239102 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:03:19.239109 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 03:03:19.239161 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 03:03:19.239259 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":349,"threads_started":1}
I20260807 03:03:19.247180 16652 raft_consensus.cc:1261] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Refusing update from remote peer abe5e680a7a24dc9a03f4c8e7c26c1f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:19.247252 16868 raft_consensus.cc:1261] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Refusing update from remote peer abe5e680a7a24dc9a03f4c8e7c26c1f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:19.247699 17355 consensus_queue.cc:1035] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0c9ed81b70b4905876c97d5e99c5828" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:19.247989 17334 consensus_queue.cc:1035] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06dd9008e4be471389f553e7a04edb7b" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:19.270025 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.265118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:40401 (local address 127.5.122.3:33743)
0807 03:03:19.265282 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:19.265287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:19.265861 (+ 574us) server_negotiation.cc:407] Connection header received
0807 03:03:19.265976 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:19.265979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:19.266036 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:19.266099 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:19.266453 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.267183 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.268482 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.268929 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.269034 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.269391 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:19.269486 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:19.269558 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:19.269683 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 03:03:19.269756 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 03:03:19.270092 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.265173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:33743 (local address 127.5.122.2:40401)
0807 03:03:19.265772 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:19.265787 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:19.265896 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:19.266105 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:19.266108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:19.266115 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 03:03:19.266338 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:19.266349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.267291 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.267295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:19.268335 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:19.268348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.269071 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.269077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:19.269211 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.269247 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 03:03:19.269573 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 03:03:19.269584 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 03:03:19.269652 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 03:03:19.269780 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":324,"threads_started":1}
I20260807 03:03:19.296926 16414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:19.297161 16414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:19.299814 16414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7bb971ca50546deb880a96b18417960: Sending DeleteTablet for 3 replicas of tablet a53dad9c192246929855c0ae994c07fc
I20260807 03:03:19.300496 16621 tablet_service.cc:1514] Processing DeleteTablet for tablet a53dad9c192246929855c0ae994c07fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:19 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:52568
I20260807 03:03:19.300590 17045 tablet_service.cc:1514] Processing DeleteTablet for tablet a53dad9c192246929855c0ae994c07fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:19 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:50634
I20260807 03:03:19.300665 16827 tablet_service.cc:1514] Processing DeleteTablet for tablet a53dad9c192246929855c0ae994c07fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:19 UTC) from {username='oryx', principal='oryx/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47840
I20260807 03:03:19.300827 17358 tablet_replica.cc:330] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: stopping tablet replica
I20260807 03:03:19.300843 17357 tablet_replica.cc:330] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: stopping tablet replica
I20260807 03:03:19.300920 17359 tablet_replica.cc:330] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: stopping tablet replica
I20260807 03:03:19.301025 17357 raft_consensus.cc:2229] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:19.301026 17358 raft_consensus.cc:2229] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:03:19.301064 17359 raft_consensus.cc:2229] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:19.301189 17359 raft_consensus.cc:2258] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:19.301200 17357 raft_consensus.cc:2258] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:19.301316 17358 raft_consensus.cc:2258] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:19.302372 17359 ts_tablet_manager.cc:1905] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:19.302518 17358 ts_tablet_manager.cc:1905] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:19.302551 17357 ts_tablet_manager.cc:1905] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:19.302567 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 16524
I20260807 03:03:19.309221 17357 ts_tablet_manager.cc:1918] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:03:19.309221 17358 ts_tablet_manager.cc:1918] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:03:19.309276 17359 ts_tablet_manager.cc:1918] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 03:03:19.309379 17357 log.cc:1196] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/a53dad9c192246929855c0ae994c07fc
I20260807 03:03:19.309432 17359 log.cc:1196] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/a53dad9c192246929855c0ae994c07fc
I20260807 03:03:19.309813 17357 ts_tablet_manager.cc:1939] T a53dad9c192246929855c0ae994c07fc P c0c9ed81b70b4905876c97d5e99c5828: Deleting consensus metadata
I20260807 03:03:19.309918 17359 ts_tablet_manager.cc:1939] T a53dad9c192246929855c0ae994c07fc P 06dd9008e4be471389f553e7a04edb7b: Deleting consensus metadata
I20260807 03:03:19.310887 16393 catalog_manager.cc:4841] TS c0c9ed81b70b4905876c97d5e99c5828 (127.5.122.1:36523): tablet a53dad9c192246929855c0ae994c07fc (table test-table [id=a9bc4248bced4fd3a152932b5e652f07]) successfully deleted
I20260807 03:03:19.310927 16395 catalog_manager.cc:4841] TS 06dd9008e4be471389f553e7a04edb7b (127.5.122.2:44659): tablet a53dad9c192246929855c0ae994c07fc (table test-table [id=a9bc4248bced4fd3a152932b5e652f07]) successfully deleted
I20260807 03:03:19.311214 17358 log.cc:1196] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/a53dad9c192246929855c0ae994c07fc
I20260807 03:03:19.311565 17358 ts_tablet_manager.cc:1939] T a53dad9c192246929855c0ae994c07fc P abe5e680a7a24dc9a03f4c8e7c26c1f1: Deleting consensus metadata
I20260807 03:03:19.312356 16397 catalog_manager.cc:4841] TS abe5e680a7a24dc9a03f4c8e7c26c1f1 (127.5.122.3:33743): tablet a53dad9c192246929855c0ae994c07fc (table test-table [id=a9bc4248bced4fd3a152932b5e652f07]) successfully deleted
Aug 07 03:03:19 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.1@KRBTEST.COM
I20260807 03:03:19.327750 17380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.306615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.5.122.1:36523)
0807 03:03:19.311070 (+ 4455us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:19.311076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:19.314423 (+ 3347us) server_negotiation.cc:407] Connection header received
0807 03:03:19.314565 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:19.314571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:19.314647 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:19.314749 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:19.316205 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.317197 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.318434 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.318853 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.318961 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.322619 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:19.322634 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:19.322649 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:19.322683 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:19.325064 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:19.325687 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:19.325694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:19.325697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:19.325842 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:19.326202 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:19.326206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:19.326208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:19.326594 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:19.326882 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:19.327171 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 03:03:19.327407 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4336,"thread_start_us":4487,"threads_started":1}
I20260807 03:03:19.328404 16722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:19.449148 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 16524
I20260807 03:03:19.475991 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 16736
Aug 07 03:03:19 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.2@KRBTEST.COM
I20260807 03:03:19.489585 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.476741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47854 (local address 127.5.122.2:44659)
0807 03:03:19.476931 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:19.476935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:19.476952 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:03:19.477090 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:19.477093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:19.477157 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:19.477241 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:19.478441 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.479158 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.480338 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.480711 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.480800 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.484521 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:19.484532 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:19.484545 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:19.484577 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:19.487330 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:19.487914 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:19.487919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:19.487922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:19.488040 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:19.488303 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:19.488306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:19.488307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:19.488632 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:19.488854 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:19.489071 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 03:03:19.489261 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:03:19.490229 16932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:19.612866 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 16736
I20260807 03:03:19.641546 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 16946
Aug 07 03:03:19 dist-test-slave-fgb3 krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.3@KRBTEST.COM
I20260807 03:03:19.655905 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:19.642368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.5.122.3:33743)
0807 03:03:19.642538 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:19.642544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:19.642587 (+ 43us) server_negotiation.cc:407] Connection header received
0807 03:03:19.642792 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:19.642799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:19.642873 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:19.642961 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:19.644257 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.644995 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.646202 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:19.646680 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:19.646796 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:19.650837 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:19.650851 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:19.650866 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:19.650899 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:19.653344 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:19.653932 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:19.653941 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:19.653945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:19.654084 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:19.654446 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:19.654452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:19.654455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:19.654797 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:19.655086 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:19.655418 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 03:03:19.655634 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:03:19.656493 17143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:19.780469 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 16946
I20260807 03:03:19.808204 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 16367
I20260807 03:03:19.809314 16474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:19.923020 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 16367
2026-08-07T03:03:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6009 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:03:20 dist-test-slave-fgb3 krb5kdc[17393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:03:20 dist-test-slave-fgb3 krb5kdc[17393](info): set up 2 sockets
Aug 07 03:03:20 dist-test-slave-fgb3 krb5kdc[17393](info): commencing operation
krb5kdc: starting...
W20260807 03:03:21.105393 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:03:21 dist-test-slave-fgb3 krb5kdc[17393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:03:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:03:21.333195 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:35111
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:35111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:21.647567 17409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:21.647835 17409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:21.647921 17409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:21.654874 17409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:21.654973 17409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:21.655018 17409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:21.655073 17409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:21.655119 17409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:21.664618 17409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:35111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:35111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:21.665748 17409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:21.666975 17409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:21.677898 17418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:21.677825 17421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:21.677863 17419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:21.678483 17409 server_base.cc:1034] running on GCE node
I20260807 03:03:21.679209 17409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:21.680569 17409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:21.681799 17409 hybrid_clock.cc:648] HybridClock initialized: now 1786071801681741 us; error 42 us; skew 500 ppm
I20260807 03:03:21.682120 17409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T03:03:21Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1822 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:03:21 dist-test-slave-fgb3 krb5kdc[17428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:03:21 dist-test-slave-fgb3 krb5kdc[17428](info): set up 2 sockets
Aug 07 03:03:21 dist-test-slave-fgb3 krb5kdc[17428](info): commencing operation
krb5kdc: starting...
W20260807 03:03:22.913220 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:03:23 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:03:23.151768 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44187
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:44187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:23.464685 17451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:23.464987 17451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:23.465124 17451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:23.472321 17451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:23.472436 17451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:23.472509 17451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:23.472594 17451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:23.472651 17451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:23.482468 17451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:44187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:23.483860 17451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:23.485118 17451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:23.496272 17458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:23.496316 17459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:23.496343 17462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:23.497087 17451 server_base.cc:1034] running on GCE node
I20260807 03:03:23.497764 17451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:23.499112 17451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:23.500334 17451 hybrid_clock.cc:648] HybridClock initialized: now 1786071803500272 us; error 50 us; skew 500 ppm
I20260807 03:03:23.500686 17451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:23 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071803, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:23.506253 17451 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:03:23.507645 17451 webserver.cc:460] Webserver started at http://127.5.122.62:46209/ using document root <none> and password file <none>
I20260807 03:03:23.508299 17451 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:23.508412 17451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:23.508710 17451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:23.511687 17451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "e4b764320bd643f29a13a8845ec2f6d9"
format_stamp: "Formatted at 2026-08-07 03:03:23 on dist-test-slave-fgb3"
server_key: "756316024b8888a12d0bcb201f8454ea"
server_key_iv: "887eafa9fdcae97b63c3db88a03ea66f"
server_key_version: "encryptionkey@0"
I20260807 03:03:23.512439 17451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "e4b764320bd643f29a13a8845ec2f6d9"
format_stamp: "Formatted at 2026-08-07 03:03:23 on dist-test-slave-fgb3"
server_key: "756316024b8888a12d0bcb201f8454ea"
server_key_iv: "887eafa9fdcae97b63c3db88a03ea66f"
server_key_version: "encryptionkey@0"
I20260807 03:03:23.517293 17451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 03:03:23.520547 17472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:23.521749 17451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:03:23.521916 17451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "e4b764320bd643f29a13a8845ec2f6d9"
format_stamp: "Formatted at 2026-08-07 03:03:23 on dist-test-slave-fgb3"
server_key: "756316024b8888a12d0bcb201f8454ea"
server_key_iv: "887eafa9fdcae97b63c3db88a03ea66f"
server_key_version: "encryptionkey@0"
I20260807 03:03:23.522090 17451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:23.537984 17451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:23.542105 17451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:23.542387 17451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:23.557785 17451 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:44187
I20260807 03:03:23.557817 17544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:44187 every 8 connection(s)
I20260807 03:03:23.559104 17451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:23.561537 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 17451
I20260807 03:03:23.561750 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:03:23.562606 5608 external_mini_cluster.cc:1428] Setting key 5f493c2861a2a28b0721e10a35ae7ec0
I20260807 03:03:23.563199 17546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:23.576722 17546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Bootstrap starting.
Aug 07 03:03:23 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:23.580233 17546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:23.581507 17546 log.cc:824] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:23.584445 17546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: No bootstrap required, opened a new log
I20260807 03:03:23.585924 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:23.564568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.5.122.62:44187)
0807 03:03:23.565293 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:23.565307 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:23.565354 (+ 47us) server_negotiation.cc:407] Connection header received
0807 03:03:23.567852 (+ 2498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:23.567880 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:23.568345 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:23.568744 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:23.570046 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:23.571967 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:23.573150 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:23.574180 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:23.574409 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:23.578139 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:23.578151 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:23.578168 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:23.578210 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:23.581903 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:23.582749 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:23.582756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:23.582761 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:23.582962 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:23.583347 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:23.583352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:23.583356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:23.583909 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:23.584186 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:23.584611 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 03:03:23.585006 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":303,"threads_started":1}
I20260807 03:03:23.592163 17546 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:23.592485 17546 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:23.592571 17546 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b764320bd643f29a13a8845ec2f6d9, State: Initialized, Role: FOLLOWER
I20260807 03:03:23.593052 17546 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:23.593261 17546 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:23.593398 17546 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:23.593571 17546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:23.595615 17546 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:23.596024 17546 leader_election.cc:302] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4b764320bd643f29a13a8845ec2f6d9; no voters:
I20260807 03:03:23.596506 17546 leader_election.cc:288] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:03:23.596625 17554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:23.597214 17554 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 LEADER]: Becoming Leader. State: Replica: e4b764320bd643f29a13a8845ec2f6d9, State: Running, Role: LEADER
I20260807 03:03:23.597688 17554 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:23.598420 17546 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:23.600126 17556 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4b764320bd643f29a13a8845ec2f6d9. Latest consensus state: current_term: 1 leader_uuid: "e4b764320bd643f29a13a8845ec2f6d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } } }
I20260807 03:03:23.600343 17555 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4b764320bd643f29a13a8845ec2f6d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } } }
I20260807 03:03:23.600595 17556 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:23.600639 17555 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:23.603436 17562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:23.608383 17562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:23.617377 17562 catalog_manager.cc:1347] Generated new cluster ID: f983c31b7d77418e8136c44402618e01
I20260807 03:03:23.617467 17562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:23.628463 17562 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:03:23.629630 17562 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:23.640646 17562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Generated new TSK 0
I20260807 03:03:23.641343 17562 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:03:23.700959 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44187
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:24.026676 17581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:24.026927 17581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:24.027052 17581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:24.033954 17581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:24.034044 17581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:24.034185 17581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:24.043308 17581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:24.044406 17581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:24.045583 17581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:24.057591 17590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:24.057516 17592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:24.057628 17589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:24.058481 17581 server_base.cc:1034] running on GCE node
I20260807 03:03:24.059052 17581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:24.059913 17581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:24.061111 17581 hybrid_clock.cc:648] HybridClock initialized: now 1786071804061071 us; error 29 us; skew 500 ppm
I20260807 03:03:24.061395 17581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:24 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071804, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:24.064365 17581 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:03:24.065738 17581 webserver.cc:460] Webserver started at http://127.5.122.1:37579/ using document root <none> and password file <none>
I20260807 03:03:24.066510 17581 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:24.066632 17581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:24.066928 17581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:24.069645 17581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "05cfab41536240feb8e12f6fea0b261a"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "323a08cf37f5e4525b2064fdcb077110"
server_key_iv: "cb7c15a6b246fb1c470892202507855e"
server_key_version: "encryptionkey@0"
I20260807 03:03:24.070425 17581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "05cfab41536240feb8e12f6fea0b261a"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "323a08cf37f5e4525b2064fdcb077110"
server_key_iv: "cb7c15a6b246fb1c470892202507855e"
server_key_version: "encryptionkey@0"
I20260807 03:03:24.075152 17581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 03:03:24.078433 17601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:24.079753 17581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:03:24.079902 17581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "05cfab41536240feb8e12f6fea0b261a"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "323a08cf37f5e4525b2064fdcb077110"
server_key_iv: "cb7c15a6b246fb1c470892202507855e"
server_key_version: "encryptionkey@0"
I20260807 03:03:24.080078 17581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:24.133435 17581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:24.137647 17581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:24.137933 17581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:24.138942 17581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:24.140327 17581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:24.140414 17581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:24.140529 17581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:24.140581 17581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:24.170035 17581 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:32933
I20260807 03:03:24.170087 17781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:32933 every 8 connection(s)
I20260807 03:03:24.171319 17581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:24.175300 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 17581
I20260807 03:03:24.175503 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:03:24.176182 5608 external_mini_cluster.cc:1428] Setting key 181022e51ddfce78710a4ed7e12d5b3a
Aug 07 03:03:24 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071804, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:24.190718 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:24.174038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:60505 (local address 127.5.122.62:44187)
0807 03:03:24.174483 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:24.174487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:24.175414 (+ 927us) server_negotiation.cc:407] Connection header received
0807 03:03:24.177235 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:24.177241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:24.177315 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:24.177419 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:24.178864 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.179571 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.181027 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.181416 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.181525 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:24.185252 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:24.185257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:24.185260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:24.185293 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:24.187742 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:24.188468 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:24.188473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:24.188477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:24.188622 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:24.189010 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:24.189015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:24.189018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:24.189352 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:24.189491 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:24.190143 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 03:03:24.190388 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":194,"threads_started":1}
I20260807 03:03:24.191958 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:24.174582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:60505)
0807 03:03:24.175251 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:24.175287 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:24.176987 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:24.177635 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:24.177643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:24.178141 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:24.178709 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:24.178725 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.179705 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.179708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:24.180896 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:24.180909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.181624 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.181631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:24.181844 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:24.182512 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:24.182533 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:24.185050 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:24.187885 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:24.187894 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:24.187910 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:24.188345 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:24.188715 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:24.188717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:24.188720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:24.188905 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:24.189470 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:24.189476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:24.189767 (+ 291us) client_negotiation.cc:769] Sending connection context
0807 03:03:24.189996 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 03:03:24.190426 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":276,"threads_started":1}
I20260807 03:03:24.193660 17784 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44187
I20260807 03:03:24.193980 17784 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:24.194723 17784 heartbeater.cc:511] Master 127.5.122.62:44187 requested a full tablet report, sending...
I20260807 03:03:24.196846 17498 ts_manager.cc:194] Registered new tserver with Master: 05cfab41536240feb8e12f6fea0b261a (127.5.122.1:32933)
I20260807 03:03:24.198841 17498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:60505
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:03:24.234086 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44187
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:24.552312 17791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:24.552564 17791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:24.552655 17791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:24.559696 17791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:24.559801 17791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:24.559948 17791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:24.569423 17791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:24.570637 17791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:24.571904 17791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:24.584347 17801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:24.584400 17799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:24.584832 17798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:24.587539 17791 server_base.cc:1034] running on GCE node
I20260807 03:03:24.588160 17791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:24.589042 17791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:24.590235 17791 hybrid_clock.cc:648] HybridClock initialized: now 1786071804590178 us; error 44 us; skew 500 ppm
I20260807 03:03:24.590551 17791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:24 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071804, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:24.593464 17791 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:03:24.594929 17791 webserver.cc:460] Webserver started at http://127.5.122.2:45093/ using document root <none> and password file <none>
I20260807 03:03:24.595583 17791 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:24.595698 17791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:24.595999 17791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:24.598910 17791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "a0c7e538108b4325bdfb2ad46d247776"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "9ba0335fd7e97bdaf354a4beabeef7f7"
server_key_iv: "c759e784596de5a64f7e165fa6e14347"
server_key_version: "encryptionkey@0"
I20260807 03:03:24.599646 17791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "a0c7e538108b4325bdfb2ad46d247776"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "9ba0335fd7e97bdaf354a4beabeef7f7"
server_key_iv: "c759e784596de5a64f7e165fa6e14347"
server_key_version: "encryptionkey@0"
I20260807 03:03:24.604655 17791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:03:24.607940 17812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:24.609164 17791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 03:03:24.609315 17791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "a0c7e538108b4325bdfb2ad46d247776"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "9ba0335fd7e97bdaf354a4beabeef7f7"
server_key_iv: "c759e784596de5a64f7e165fa6e14347"
server_key_version: "encryptionkey@0"
I20260807 03:03:24.609494 17791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:24.626832 17791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:24.630739 17791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:24.631016 17791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:24.631969 17791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:24.633323 17791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:24.633402 17791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:24.633529 17791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:24.633582 17791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:24.663466 17791 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:33011
I20260807 03:03:24.663532 17986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:33011 every 8 connection(s)
I20260807 03:03:24.664700 17791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:24.669382 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 17791
I20260807 03:03:24.669528 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:03:24.669967 5608 external_mini_cluster.cc:1428] Setting key b18a1975fdc351f0d97e8e9481c4dddd
Aug 07 03:03:24 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071804, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:24.683056 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:24.667031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:42709 (local address 127.5.122.62:44187)
0807 03:03:24.667191 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:24.667195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:24.668262 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 03:03:24.670077 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:24.670084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:24.670163 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:24.670247 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:24.671878 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.672578 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.674078 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.674552 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.674659 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:24.677949 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:24.677954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:24.677957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:24.677999 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:24.680295 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:24.680933 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:24.680938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:24.680942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:24.681070 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:24.681456 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:24.681461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:24.681464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:24.681715 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:24.681842 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:24.682496 (+ 654us) server_negotiation.cc:299] Negotiation successful
0807 03:03:24.682723 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:03:24.684002 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:24.667421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:42709)
0807 03:03:24.668094 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:24.668137 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:24.669808 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:24.670513 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:24.670521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:24.671119 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:24.671722 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:24.671742 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.672722 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.672726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:24.673939 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:24.673951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:24.674763 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:24.674771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:24.674986 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:24.675578 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:24.675603 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:24.677741 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:24.680418 (+ 2677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:24.680427 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:24.680441 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:24.680825 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:24.681170 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:24.681175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:24.681179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:24.681352 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:24.681834 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:24.681842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:24.682137 (+ 295us) client_negotiation.cc:769] Sending connection context
0807 03:03:24.682401 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 03:03:24.682869 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":259,"threads_started":1}
I20260807 03:03:24.685334 17989 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44187
I20260807 03:03:24.685709 17989 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:24.686556 17989 heartbeater.cc:511] Master 127.5.122.62:44187 requested a full tablet report, sending...
I20260807 03:03:24.688091 17498 ts_manager.cc:194] Registered new tserver with Master: a0c7e538108b4325bdfb2ad46d247776 (127.5.122.2:33011)
I20260807 03:03:24.689154 17498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:42709
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:03:24.723549 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44187
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:25.035467 17998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:25.035754 17998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:25.035866 17998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:25.043217 17998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:25.043336 17998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:25.043459 17998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:25.053391 17998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:44187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:25.054883 17998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:25.056120 17998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:25.068883 18006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:25.068955 18007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:25.068989 18013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:25.069841 17998 server_base.cc:1034] running on GCE node
I20260807 03:03:25.070513 17998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:25.071467 17998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:25.072700 17998 hybrid_clock.cc:648] HybridClock initialized: now 1786071805072632 us; error 53 us; skew 500 ppm
I20260807 03:03:25.073031 17998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:25 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071805, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:25.076556 17998 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:03:25.078049 17998 webserver.cc:460] Webserver started at http://127.5.122.3:44439/ using document root <none> and password file <none>
I20260807 03:03:25.078743 17998 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:25.078866 17998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:25.079180 17998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:25.082047 17998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "3a1f46e6d5af4265aded840f5d36deaa"
format_stamp: "Formatted at 2026-08-07 03:03:25 on dist-test-slave-fgb3"
server_key: "83093b46270c3b109a39c485fca88ede"
server_key_iv: "ce2aac2dacefe4a6dbccef7650bf105f"
server_key_version: "encryptionkey@0"
I20260807 03:03:25.082835 17998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "3a1f46e6d5af4265aded840f5d36deaa"
format_stamp: "Formatted at 2026-08-07 03:03:25 on dist-test-slave-fgb3"
server_key: "83093b46270c3b109a39c485fca88ede"
server_key_iv: "ce2aac2dacefe4a6dbccef7650bf105f"
server_key_version: "encryptionkey@0"
I20260807 03:03:25.088029 17998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:03:25.091322 18023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:25.092553 17998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:03:25.092705 17998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "3a1f46e6d5af4265aded840f5d36deaa"
format_stamp: "Formatted at 2026-08-07 03:03:25 on dist-test-slave-fgb3"
server_key: "83093b46270c3b109a39c485fca88ede"
server_key_iv: "ce2aac2dacefe4a6dbccef7650bf105f"
server_key_version: "encryptionkey@0"
I20260807 03:03:25.092913 17998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:25.104126 17998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:25.107692 17998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:25.107947 17998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:25.108798 17998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:25.110059 17998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:25.110134 17998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:25.110248 17998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:25.110301 17998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:25.139784 17998 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36375
I20260807 03:03:25.139827 18199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36375 every 8 connection(s)
I20260807 03:03:25.140998 17998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:25.142890 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 17998
I20260807 03:03:25.143069 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:03:25.143639 5608 external_mini_cluster.cc:1428] Setting key a923116c0d26113ab013eeafd682a4f4
Aug 07 03:03:25 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071805, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:25.159381 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:25.143495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:43313 (local address 127.5.122.62:44187)
0807 03:03:25.143653 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:25.143657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:25.145054 (+ 1397us) server_negotiation.cc:407] Connection header received
0807 03:03:25.146885 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:25.146889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:25.146955 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:25.147032 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:25.148398 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.149130 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.150600 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.150995 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.151080 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:25.154131 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:25.154136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:25.154139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:25.154169 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:25.156498 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.157113 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.157118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.157121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.157283 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.157680 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.157685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.157688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.158020 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:25.158174 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:25.158970 (+ 796us) server_negotiation.cc:299] Negotiation successful
0807 03:03:25.159131 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 03:03:25.160264 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:25.144104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:43313)
0807 03:03:25.144891 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:25.144941 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:25.146657 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:25.147224 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:25.147232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:25.147732 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:25.148271 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:25.148288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.149237 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.149240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:25.150494 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:25.150511 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.151157 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.151162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:25.151350 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:25.151920 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:25.151941 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:25.153938 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:25.156609 (+ 2671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:25.156615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:25.156625 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:25.157010 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:25.157399 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:25.157404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:25.157407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:25.157590 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:25.158154 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:25.158161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:25.158606 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 03:03:25.158888 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 03:03:25.159258 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":313,"threads_started":1}
I20260807 03:03:25.161446 18209 heartbeater.cc:347] Connected to a master server at 127.5.122.62:44187
I20260807 03:03:25.161749 18209 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:25.162487 18209 heartbeater.cc:511] Master 127.5.122.62:44187 requested a full tablet report, sending...
I20260807 03:03:25.163766 17498 ts_manager.cc:194] Registered new tserver with Master: 3a1f46e6d5af4265aded840f5d36deaa (127.5.122.3:36375)
I20260807 03:03:25.164518 17498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:43313
I20260807 03:03:25.171031 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.1@KRBTEST.COM" created.
I20260807 03:03:25.202483 17784 heartbeater.cc:503] Master 127.5.122.62:44187 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab.
Entry for principal oryx/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab.
WARNING: no policy specified for oryx/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab.
Entry for principal oryx/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab.
WARNING: no policy specified for oryx/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab.
Entry for principal oryx/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab.
I20260807 03:03:25.333455 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 17581
Aug 07 03:03:25 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:03:25.351936 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:25.334838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.5.122.1:32933)
0807 03:03:25.335488 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:25.335498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:25.335530 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:03:25.335607 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:25.335616 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:25.335886 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:25.336067 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:25.339577 (+ 3510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.340606 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.342103 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.342870 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.342971 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:25.346988 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:25.347000 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:25.347015 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:25.347051 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:25.349553 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.350136 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.350143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.350145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.350276 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.350598 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.350601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.350603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.350941 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:25.351164 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:25.351378 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 03:03:25.351576 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":305,"threads_started":1}
I20260807 03:03:25.353188 17775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:25.435504 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 17581
I20260807 03:03:25.457161 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 17791
Aug 07 03:03:25 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:03:25.472319 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:25.458022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.5.122.2:33011)
0807 03:03:25.458552 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:25.458559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:25.458584 (+ 25us) server_negotiation.cc:407] Connection header received
0807 03:03:25.458644 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:25.458650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:25.458849 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:25.458984 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:25.460312 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.461274 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.462651 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.463195 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.463287 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:25.467298 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:25.467311 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:25.467324 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:25.467356 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:25.470000 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.470635 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.470642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.470644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.470758 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.471031 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.471034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.471036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.471345 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:25.471571 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:25.471833 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 03:03:25.472032 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":244,"threads_started":1}
I20260807 03:03:25.473479 17982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:25.554956 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 17791
I20260807 03:03:25.576084 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 17998
Aug 07 03:03:25 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:03:25.592482 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:25.577044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.5.122.3:36375)
0807 03:03:25.577657 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:25.577664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:25.577693 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:03:25.577754 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:25.577760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:25.577985 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:25.578122 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:25.579500 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.580537 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.581835 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:25.582436 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:25.582529 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:25.586905 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:25.586918 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:25.586932 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:25.586970 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:25.589740 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.590637 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.590645 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.590647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.590791 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:25.591115 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:25.591118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:25.591119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:25.591447 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:25.591686 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:25.591907 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 03:03:25.592120 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":289,"threads_started":1}
I20260807 03:03:25.593746 18192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:25.675042 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 17998
I20260807 03:03:25.697121 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 17451
I20260807 03:03:25.698347 17540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:25.799373 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 17451
I20260807 03:03:25.822914 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44187
--webserver_interface=127.5.122.62
--webserver_port=46209
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:44187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:26.140595 18233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:26.140873 18233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:26.141005 18233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:26.148025 18233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:26.148128 18233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:26.148202 18233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:26.148278 18233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:26.148336 18233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:26.157914 18233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:44187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:44187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=46209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:26.159212 18233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:26.160499 18233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:26.171362 18248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:26.171432 18243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:26.171432 18246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:26.172506 18233 server_base.cc:1034] running on GCE node
I20260807 03:03:26.173242 18233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:26.174662 18233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:26.175876 18233 hybrid_clock.cc:648] HybridClock initialized: now 1786071806175820 us; error 30 us; skew 500 ppm
I20260807 03:03:26.176193 18233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071806, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:26.179136 18233 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:03:26.180478 18233 webserver.cc:460] Webserver started at http://127.5.122.62:46209/ using document root <none> and password file <none>
I20260807 03:03:26.181128 18233 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:26.181254 18233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:26.186412 18233 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:03:26.189136 18257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:26.190402 18233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:26.190583 18233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "e4b764320bd643f29a13a8845ec2f6d9"
format_stamp: "Formatted at 2026-08-07 03:03:23 on dist-test-slave-fgb3"
server_key: "756316024b8888a12d0bcb201f8454ea"
server_key_iv: "887eafa9fdcae97b63c3db88a03ea66f"
server_key_version: "encryptionkey@0"
I20260807 03:03:26.191517 18233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:26.210119 18233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:26.213953 18233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:26.214201 18233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:26.230970 18233 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:44187
I20260807 03:03:26.230998 18345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:44187 every 8 connection(s)
I20260807 03:03:26.232455 18233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:26.235780 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 18233
I20260807 03:03:26.236725 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:32933
--local_ip_for_outbound_sockets=127.5.122.1
--tserver_master_addrs=127.5.122.62:44187
--webserver_port=37579
--webserver_interface=127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:03:26.240589 18346 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:03:26.244720 18346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Bootstrap starting.
I20260807 03:03:26.263815 18346 log.cc:824] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:26.272588 18346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:03:26.273072 18346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Bootstrap complete.
I20260807 03:03:26.281864 18346 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:26.282722 18346 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b764320bd643f29a13a8845ec2f6d9, State: Initialized, Role: FOLLOWER
I20260807 03:03:26.283313 18346 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:26.283541 18346 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:26.283651 18346 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:26.283810 18346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 1 FOLLOWER]: Advancing to term 2
I20260807 03:03:26.287511 18346 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:26.288024 18346 leader_election.cc:302] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4b764320bd643f29a13a8845ec2f6d9; no voters:
I20260807 03:03:26.288602 18346 leader_election.cc:288] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 03:03:26.288743 18351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 03:03:26.289925 18351 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [term 2 LEADER]: Becoming Leader. State: Replica: e4b764320bd643f29a13a8845ec2f6d9, State: Running, Role: LEADER
I20260807 03:03:26.290521 18351 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } }
I20260807 03:03:26.290764 18346 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:26.292946 18353 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4b764320bd643f29a13a8845ec2f6d9. Latest consensus state: current_term: 2 leader_uuid: "e4b764320bd643f29a13a8845ec2f6d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } } }
I20260807 03:03:26.293303 18353 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:26.293397 18352 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e4b764320bd643f29a13a8845ec2f6d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b764320bd643f29a13a8845ec2f6d9" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 44187 } } }
I20260807 03:03:26.293744 18352 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:26.303910 18364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:26.309191 18364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:26.310647 18364 catalog_manager.cc:1259] Loaded cluster ID: f983c31b7d77418e8136c44402618e01
I20260807 03:03:26.310786 18364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:26.314823 18364 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:26.316362 18364 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e4b764320bd643f29a13a8845ec2f6d9: Loaded TSK: 0
I20260807 03:03:26.317929 18364 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 03:03:26.589855 18349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:26.590158 18349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:26.590335 18349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:26.598137 18349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:26.598263 18349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:26.598460 18349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:26.608589 18349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.1.keytab
--principal=oryx/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:32933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=37579
--tserver_master_addrs=127.5.122.62:44187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:26.609992 18349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:26.611505 18349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:26.626108 18387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:26.626127 18390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:26.628093 18385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:26.628485 18349 server_base.cc:1034] running on GCE node
I20260807 03:03:26.629074 18349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:26.630040 18349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:26.631256 18349 hybrid_clock.cc:648] HybridClock initialized: now 1786071806631203 us; error 42 us; skew 500 ppm
I20260807 03:03:26.631557 18349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071806, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:26.634595 18349 init.cc:377] Logged in from keytab as oryx/127.5.122.1@KRBTEST.COM (short username oryx)
I20260807 03:03:26.636075 18349 webserver.cc:460] Webserver started at http://127.5.122.1:37579/ using document root <none> and password file <none>
I20260807 03:03:26.636754 18349 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:26.636878 18349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:26.642500 18349 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 03:03:26.645243 18398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:26.646658 18349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:03:26.646816 18349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "05cfab41536240feb8e12f6fea0b261a"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "323a08cf37f5e4525b2064fdcb077110"
server_key_iv: "cb7c15a6b246fb1c470892202507855e"
server_key_version: "encryptionkey@0"
I20260807 03:03:26.647768 18349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:26.660135 18349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:26.664472 18349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:26.664729 18349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:26.665700 18349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:26.667115 18349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:26.667204 18349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:26.667297 18349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:26.667371 18349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:26.698629 18349 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:32933
I20260807 03:03:26.698698 18562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:32933 every 8 connection(s)
I20260807 03:03:26.700043 18349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:26.704083 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 18349
I20260807 03:03:26.704661 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:33011
--local_ip_for_outbound_sockets=127.5.122.2
--tserver_master_addrs=127.5.122.62:44187
--webserver_port=45093
--webserver_interface=127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:26.717706 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:26.702717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:48811)
0807 03:03:26.703530 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:26.703584 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:26.705356 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:26.706917 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:26.706924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:26.707433 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:26.707941 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:26.707958 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.709951 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.709954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:26.711181 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:26.711193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.712414 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.712418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:26.712601 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:26.713192 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:26.713215 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:26.716744 (+ 3529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":304,"threads_started":1}
W20260807 03:03:26.718226 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:26.702711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:48811 (local address 127.5.122.62:44187)
0807 03:03:26.703390 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:26.703401 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:26.703723 (+ 322us) server_negotiation.cc:407] Connection header received
0807 03:03:26.705946 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:26.705966 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:26.706433 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:26.706804 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:26.708135 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.709812 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.711319 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.712227 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.712414 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:26.717119 (+ 4705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:48811: BlockingRecv error: recv got EOF from 127.5.122.1:48811 (error 108)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":250,"threads_started":1}
W20260807 03:03:26.718760 18569 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:44187 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.122.62:44187: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:26.726899 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:26.719275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:51437)
0807 03:03:26.719529 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:26.719547 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:26.719643 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:26.719936 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:26.719939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:26.720237 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:26.720384 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:26.720391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.721642 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.721645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:26.722551 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:26.722562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.723328 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.723332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:26.723453 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:26.723962 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:26.723978 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:26.726697 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141}
W20260807 03:03:26.727154 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:26.719500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:51437 (local address 127.5.122.62:44187)
0807 03:03:26.719684 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:26.719689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:26.719706 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:03:26.719754 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:26.719758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:26.719822 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:26.719908 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:26.720517 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.721532 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.722675 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.723200 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.723346 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:26.726934 (+ 3588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:51437: BlockingRecv error: recv got EOF from 127.5.122.1:51437 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:26 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:26.735466 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:26.727485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:38197)
0807 03:03:26.728062 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:26.728077 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:26.728171 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:26.728462 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:26.728465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:26.728797 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:26.728992 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:26.728999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.730257 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.730260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:26.731188 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:26.731199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.732082 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.732087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:26.732192 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:26.732729 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:26.732746 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:26.735228 (+ 2482us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":273,"threads_started":1}
W20260807 03:03:26.735707 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:26.727717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:38197 (local address 127.5.122.62:44187)
0807 03:03:26.727874 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:26.727879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:26.728148 (+ 269us) server_negotiation.cc:407] Connection header received
0807 03:03:26.728258 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:26.728264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:26.728356 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:26.728455 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:26.729120 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.730143 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.731326 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:26.731936 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:26.732065 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:26.735467 (+ 3402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:38197: BlockingRecv error: recv got EOF from 127.5.122.1:38197 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 03:03:26.735852 18569 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 03:03:27.025199 18575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:27.025487 18575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:27.025580 18575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:27.032691 18575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:27.032801 18575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:27.032930 18575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:27.042480 18575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.2.keytab
--principal=oryx/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:33011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=45093
--tserver_master_addrs=127.5.122.62:44187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:27.043749 18575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:27.044992 18575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:27.057535 18591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:27.057643 18584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:27.057614 18586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:27.058521 18575 server_base.cc:1034] running on GCE node
I20260807 03:03:27.059170 18575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:27.060092 18575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:27.061298 18575 hybrid_clock.cc:648] HybridClock initialized: now 1786071807061245 us; error 45 us; skew 500 ppm
I20260807 03:03:27.061609 18575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071807, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:27.064668 18575 init.cc:377] Logged in from keytab as oryx/127.5.122.2@KRBTEST.COM (short username oryx)
I20260807 03:03:27.066080 18575 webserver.cc:460] Webserver started at http://127.5.122.2:45093/ using document root <none> and password file <none>
I20260807 03:03:27.066771 18575 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:27.066901 18575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:27.072146 18575 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 03:03:27.074704 18603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:27.075934 18575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:03:27.076067 18575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "a0c7e538108b4325bdfb2ad46d247776"
format_stamp: "Formatted at 2026-08-07 03:03:24 on dist-test-slave-fgb3"
server_key: "9ba0335fd7e97bdaf354a4beabeef7f7"
server_key_iv: "c759e784596de5a64f7e165fa6e14347"
server_key_version: "encryptionkey@0"
I20260807 03:03:27.076960 18575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:27.089118 18575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:27.093197 18575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:27.093420 18575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:27.094403 18575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:27.095773 18575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:27.095867 18575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:27.095961 18575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:27.096024 18575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:27.126072 18575 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:33011
I20260807 03:03:27.126144 18776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:33011 every 8 connection(s)
I20260807 03:03:27.127535 18575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:27.131124 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 18575
I20260807 03:03:27.131834 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:36375
--local_ip_for_outbound_sockets=127.5.122.3
--tserver_master_addrs=127.5.122.62:44187
--webserver_port=44439
--webserver_interface=127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 03:03:27.142874 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:27.129941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:38467 (local address 127.5.122.62:44187)
0807 03:03:27.130116 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.130121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.131283 (+ 1162us) server_negotiation.cc:407] Connection header received
0807 03:03:27.133206 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.133212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.133291 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.133386 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.134907 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.135613 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.137119 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.137535 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.137644 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.142625 (+ 4981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:38467: BlockingRecv error: recv got EOF from 127.5.122.2:38467 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:03:27.143529 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.130416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:38467)
0807 03:03:27.131132 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:27.131169 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:27.132938 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:27.133628 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:27.133637 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:27.134131 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:27.134728 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.134746 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.135769 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.135772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.136976 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.136988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.137748 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.137756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.137980 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.138688 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:27.138714 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:27.142433 (+ 3719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":280,"threads_started":1}
W20260807 03:03:27.144153 18778 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:44187 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.122.62:44187: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:27.152355 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.144711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:48859)
0807 03:03:27.144885 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:27.144903 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:27.144990 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:27.145246 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:27.145249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:27.145540 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:27.145667 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.145674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.146943 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.146947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.147880 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.147893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.148744 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.148749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.148869 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.149393 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:27.149410 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:27.152155 (+ 2745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 03:03:27.152661 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:27.144823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:48859 (local address 127.5.122.62:44187)
0807 03:03:27.144991 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.144996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.145016 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:03:27.145064 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.145068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.145134 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.145221 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.145812 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.146781 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.148022 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.148607 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.148721 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.152418 (+ 3697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:48859: BlockingRecv error: recv got EOF from 127.5.122.2:48859 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:27.160785 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.153165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:37597)
0807 03:03:27.153413 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:27.153428 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:27.153502 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:27.153766 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:27.153769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:27.154024 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:27.154182 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.154191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.155512 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.155516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.156403 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.156414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.157274 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.157279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.157403 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.157930 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:27.157947 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:27.160582 (+ 2635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148}
W20260807 03:03:27.161039 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:27.153296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:37597 (local address 127.5.122.62:44187)
0807 03:03:27.153480 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.153485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.153504 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:03:27.153580 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.153586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.153655 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.153761 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.154307 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.155351 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.156539 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.157122 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.157251 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.160826 (+ 3575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:37597: BlockingRecv error: recv got EOF from 127.5.122.2:37597 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 03:03:27.161203 18778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 03:03:27.455097 18783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:27.455361 18783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:27.455494 18783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:27.462656 18783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:27.462760 18783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:27.462917 18783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:27.472452 18783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/krb5kdc/oryx_127.5.122.3.keytab
--principal=oryx/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:36375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=44439
--tserver_master_addrs=127.5.122.62:44187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:27.473654 18783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:27.474962 18783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:27.487556 18791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:27.487597 18795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:27.487623 18792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:27.488341 18783 server_base.cc:1034] running on GCE node
I20260807 03:03:27.488893 18783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:27.489719 18783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:27.490914 18783 hybrid_clock.cc:648] HybridClock initialized: now 1786071807490865 us; error 43 us; skew 500 ppm
I20260807 03:03:27.491190 18783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071807, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:27.494166 18783 init.cc:377] Logged in from keytab as oryx/127.5.122.3@KRBTEST.COM (short username oryx)
I20260807 03:03:27.495586 18783 webserver.cc:460] Webserver started at http://127.5.122.3:44439/ using document root <none> and password file <none>
I20260807 03:03:27.496291 18783 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:27.496418 18783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:27.501374 18783 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 03:03:27.503907 18804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:27.505089 18783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 03:03:27.505223 18783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "3a1f46e6d5af4265aded840f5d36deaa"
format_stamp: "Formatted at 2026-08-07 03:03:25 on dist-test-slave-fgb3"
server_key: "83093b46270c3b109a39c485fca88ede"
server_key_iv: "ce2aac2dacefe4a6dbccef7650bf105f"
server_key_version: "encryptionkey@0"
I20260807 03:03:27.506114 18783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:27.536792 18783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:27.540625 18783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:27.540855 18783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:27.541715 18783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:27.542944 18783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:27.543031 18783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:27.543136 18783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:27.543203 18783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:27.572074 18783 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36375
I20260807 03:03:27.572139 18960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36375 every 8 connection(s)
I20260807 03:03:27.573417 18783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:27.577486 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 18783
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 03:03:27.590780 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:27.575821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:36439 (local address 127.5.122.62:44187)
0807 03:03:27.575999 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.576004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.577085 (+ 1081us) server_negotiation.cc:407] Connection header received
0807 03:03:27.579285 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.579292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.579390 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.579508 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.581167 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.582401 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.584491 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.585035 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.585146 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.590509 (+ 5363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:36439: BlockingRecv error: recv got EOF from 127.5.122.3:36439 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:03:27.591787 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.576262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:36439)
0807 03:03:27.576938 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:27.576975 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:27.579034 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:27.579729 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:27.579737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:27.580282 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:27.580940 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.580964 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.582576 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.582581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.584363 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.584385 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.585227 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.585232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.585530 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.586049 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:27.586078 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:27.590264 (+ 4186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":256,"threads_started":1}
W20260807 03:03:27.592536 18961 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:44187 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.122.62:44187: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
I20260807 03:03:27.594739 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.578435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.5.122.62:44187)
0807 03:03:27.578843 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.578849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.579165 (+ 316us) server_negotiation.cc:407] Connection header received
0807 03:03:27.579329 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.579332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.579394 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.579545 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.581355 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.582436 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.584138 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.584580 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.584688 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.587798 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:27.587809 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:27.587827 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:27.587875 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:27.591459 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:27.592171 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:27.592178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:27.592183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:27.592389 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:27.592768 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:27.592773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:27.592776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:27.593333 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:27.593654 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:27.594037 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 03:03:27.594283 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":196,"threads_started":1}
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:27.600613 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.593051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:57253)
0807 03:03:27.593212 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:27.593231 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:27.593325 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:27.593816 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:27.593820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:27.594208 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:27.594405 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.594418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.595595 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.595601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.596619 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.596633 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.597367 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.597373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.597548 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.598073 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:27.598090 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:27.600437 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 03:03:27.600783 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:27.593164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:57253 (local address 127.5.122.62:44187)
0807 03:03:27.593295 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.593299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.593313 (+ 14us) server_negotiation.cc:407] Connection header received
0807 03:03:27.593431 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.593435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.593498 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.593631 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.594530 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.595458 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.596746 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.597222 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.597346 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.600600 (+ 3254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:57253: BlockingRecv error: recv got EOF from 127.5.122.3:57253 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:27.607803 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.601153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:37277)
0807 03:03:27.601290 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:27.601304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:27.601398 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:27.601701 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:27.601706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:27.601997 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:27.602131 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.602139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.603108 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.603113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.604008 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.604019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.604624 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.604630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.604788 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.605254 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:27.605269 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:27.607539 (+ 2270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 03:03:27.607978 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:27.601265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:37277 (local address 127.5.122.62:44187)
0807 03:03:27.601484 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.601489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.601504 (+ 15us) server_negotiation.cc:407] Connection header received
0807 03:03:27.601539 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.601543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.601588 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.601659 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.602257 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.602983 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.604134 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.604501 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.604599 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.607733 (+ 3134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:37277: BlockingRecv error: recv got EOF from 127.5.122.3:37277 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
W20260807 03:03:27.608196 18961 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:27 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:27.744508 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:27.736419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:58301)
0807 03:03:27.737021 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:27.737041 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:27.737180 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:27.737526 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:27.737531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:27.738163 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:27.738442 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.738455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.739467 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.739471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.740401 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:27.740411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.741037 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.741041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:27.741160 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.741678 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:27.741695 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:27.744280 (+ 2585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":308,"threads_started":1}
W20260807 03:03:27.744647 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:27.736627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:58301 (local address 127.5.122.62:44187)
0807 03:03:27.736852 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:27.736858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:27.737129 (+ 271us) server_negotiation.cc:407] Connection header received
0807 03:03:27.737295 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:27.737300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:27.737374 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:27.737451 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:27.738594 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.739331 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.740505 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:27.740910 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:27.740994 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:27.744464 (+ 3470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:58301: BlockingRecv error: recv got EOF from 127.5.122.1:58301 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 03:03:28 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:28 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:28.169739 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:28.161650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:36939)
0807 03:03:28.162170 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:28.162187 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:28.162293 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:28.162638 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:28.162643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:28.163131 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:28.163342 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:28.163352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.164234 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.164237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:28.165186 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:28.165198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.165819 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.165823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:28.165943 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:28.166480 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:28.166496 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:28.169425 (+ 2929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":283,"threads_started":1}
W20260807 03:03:28.169914 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:28.161708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:36939 (local address 127.5.122.62:44187)
0807 03:03:28.161873 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:28.161878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:28.162247 (+ 369us) server_negotiation.cc:407] Connection header received
0807 03:03:28.162429 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:28.162434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:28.162493 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:28.162572 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:28.163449 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.164130 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.165297 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.165689 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.165774 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:28.169643 (+ 3869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:36939: BlockingRecv error: recv got EOF from 127.5.122.2:36939 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 03:03:28 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:28 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:28.625047 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:28.608771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:39667)
0807 03:03:28.609256 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:28.609273 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:28.609412 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:28.609720 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:28.609725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:28.610371 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:28.610642 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:28.610656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.611906 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.611909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:28.612853 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:28.612864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.613484 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.613488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:28.613599 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:28.621819 (+ 8220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:28.621842 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:28.624821 (+ 2979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":271,"threads_started":1}
W20260807 03:03:28.625205 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:28.608825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:39667 (local address 127.5.122.62:44187)
0807 03:03:28.609015 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:28.609022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:28.609358 (+ 336us) server_negotiation.cc:407] Connection header received
0807 03:03:28.609507 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:28.609511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:28.609574 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:28.609660 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:28.610780 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.611703 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.612958 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.613340 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.613428 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:28.625011 (+ 11583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:39667: BlockingRecv error: recv got EOF from 127.5.122.3:39667 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 03:03:28 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:28 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:28.753451 18970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:28.745341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:43317)
0807 03:03:28.745862 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:28.745877 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:28.745992 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:28.746244 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:28.746246 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:28.746669 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:28.746827 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:28.746836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.747787 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.747790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:28.748985 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:28.749001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.749853 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.749857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:28.749971 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:28.750519 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:28.750536 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:28.753208 (+ 2672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":276,"threads_started":1}
W20260807 03:03:28.753616 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:28.745411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:43317 (local address 127.5.122.62:44187)
0807 03:03:28.745575 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:28.745579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:28.745944 (+ 365us) server_negotiation.cc:407] Connection header received
0807 03:03:28.746075 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:28.746079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:28.746138 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:28.746211 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:28.746942 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.747695 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.749117 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:28.749703 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:28.749809 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:28.753394 (+ 3585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:43317: BlockingRecv error: recv got EOF from 127.5.122.1:43317 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 03:03:29 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:29 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:29.178227 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:29.170736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:41053)
0807 03:03:29.171200 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:29.171216 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:29.171327 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:29.171602 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:29.171605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:29.172000 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:29.172146 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:29.172154 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.173096 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.173099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:29.173975 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:29.173985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.174633 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.174637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:29.174742 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:29.175236 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:29.175251 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:29.177875 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":243,"threads_started":1}
W20260807 03:03:29.178355 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:29.170805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:41053 (local address 127.5.122.62:44187)
0807 03:03:29.171000 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:29.171007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:29.171289 (+ 282us) server_negotiation.cc:407] Connection header received
0807 03:03:29.171420 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:29.171424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:29.171485 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:29.171558 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:29.172263 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.172993 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.174075 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.174526 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.174614 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:29.178119 (+ 3505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:41053: BlockingRecv error: recv got EOF from 127.5.122.2:41053 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 03:03:29 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:29 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:29.634243 18973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:29.625958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:55145)
0807 03:03:29.626503 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:29.626519 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:29.626648 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:29.626961 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:29.626966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:29.627391 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:29.627537 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:29.627545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.628593 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.628597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:29.629535 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:29.629547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.630249 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.630254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:29.630451 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:29.630980 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:29.630997 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:29.633916 (+ 2919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":275,"threads_started":1}
W20260807 03:03:29.634424 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:29.626118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:55145 (local address 127.5.122.62:44187)
0807 03:03:29.626345 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:29.626352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:29.626595 (+ 243us) server_negotiation.cc:407] Connection header received
0807 03:03:29.626760 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:29.626764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:29.626823 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:29.626901 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:29.627681 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.628437 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.629665 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.630102 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.630210 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:29.634123 (+ 3913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:55145: BlockingRecv error: recv got EOF from 127.5.122.3:55145 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 03:03:29 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:29 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:29.762389 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:29.754258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:58363)
0807 03:03:29.754785 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:29.754800 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:29.754938 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:29.755239 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:29.755244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:29.755619 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:29.755791 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:29.755799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.756796 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.756801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:29.757709 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:29.757720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.758410 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.758416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:29.758597 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:29.759143 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:29.759162 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:29.762016 (+ 2854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":285,"threads_started":1}
W20260807 03:03:29.762486 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:29.754356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:58363 (local address 127.5.122.62:44187)
0807 03:03:29.754531 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:29.754538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:29.754881 (+ 343us) server_negotiation.cc:407] Connection header received
0807 03:03:29.755052 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:29.755055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:29.755118 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:29.755197 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:29.755935 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.756657 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.757845 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:29.758250 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:29.758402 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:29.762238 (+ 3836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:58363: BlockingRecv error: recv got EOF from 127.5.122.1:58363 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 03:03:30 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:30 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:30.186954 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:30.179134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:60747)
0807 03:03:30.179559 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:30.179575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:30.179716 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:30.180045 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:30.180049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:30.180422 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:30.180600 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:30.180610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.181664 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.181668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:30.182627 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:30.182640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.183321 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.183328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:30.183500 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:30.184013 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:30.184029 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:30.186682 (+ 2653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":232,"threads_started":1}
W20260807 03:03:30.187305 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:30.179204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:60747 (local address 127.5.122.62:44187)
0807 03:03:30.179408 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:30.179414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:30.179641 (+ 227us) server_negotiation.cc:407] Connection header received
0807 03:03:30.179836 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:30.179841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:30.179913 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:30.179993 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:30.180754 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.181536 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.182755 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.183182 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.183288 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:30.186936 (+ 3648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:60747: BlockingRecv error: recv got EOF from 127.5.122.2:60747 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 03:03:30 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:30 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:30.642925 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:30.635148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:45441)
0807 03:03:30.635597 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:30.635614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:30.635726 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:30.636001 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:30.636004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:30.636347 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:30.636517 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:30.636525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.637413 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.637416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:30.638343 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:30.638356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.638934 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.638937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:30.639062 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:30.639573 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:30.639590 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:30.642598 (+ 3008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":240,"threads_started":1}
W20260807 03:03:30.643018 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:30.635246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:45441 (local address 127.5.122.62:44187)
0807 03:03:30.635470 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:30.635476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:30.635684 (+ 208us) server_negotiation.cc:407] Connection header received
0807 03:03:30.635812 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:30.635815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:30.635897 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:30.635971 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:30.636618 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.637318 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.638444 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.638827 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.638909 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:30.642826 (+ 3917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:45441: BlockingRecv error: recv got EOF from 127.5.122.3:45441 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 03:03:30 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:30 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:30.777323 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:30.763326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:33593)
0807 03:03:30.763820 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:30.763835 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:30.763985 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:30.764317 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:30.764322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:30.764702 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:30.764865 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:30.764885 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.766191 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.766197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:30.767735 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:30.767752 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.768679 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.768683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:30.768867 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:30.773598 (+ 4731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:30.773624 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:30.777056 (+ 3432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":266,"threads_started":1}
W20260807 03:03:30.777716 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:30.763416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:33593 (local address 127.5.122.62:44187)
0807 03:03:30.763627 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:30.763634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:30.763934 (+ 300us) server_negotiation.cc:407] Connection header received
0807 03:03:30.764094 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:30.764100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:30.764171 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:30.764251 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:30.765037 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.766044 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.767889 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:30.768521 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:30.768670 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:30.777333 (+ 8663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:33593: BlockingRecv error: recv got EOF from 127.5.122.1:33593 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 03:03:31 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:31 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:31.195860 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:31.187916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:38063)
0807 03:03:31.188344 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:31.188360 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:31.188478 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:31.188746 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:31.188749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:31.189119 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:31.189270 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:31.189279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.190158 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.190161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:31.191248 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:31.191262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.191868 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.191881 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:31.192011 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:31.192592 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:31.192610 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:31.195563 (+ 2953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":227,"threads_started":1}
W20260807 03:03:31.195971 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:31.187977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:38063 (local address 127.5.122.62:44187)
0807 03:03:31.188187 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:31.188194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:31.188435 (+ 241us) server_negotiation.cc:407] Connection header received
0807 03:03:31.188563 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:31.188566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:31.188629 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:31.188702 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:31.189370 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.190065 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.191345 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.191760 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.191845 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:31.195717 (+ 3872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:38063: BlockingRecv error: recv got EOF from 127.5.122.2:38063 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 03:03:31 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:31 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:31.651595 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:31.643763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:34275)
0807 03:03:31.644218 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:31.644235 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:31.644355 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:31.644609 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:31.644612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:31.644964 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:31.645113 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:31.645122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.646196 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.646200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:31.647193 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:31.647207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.647898 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.647904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:31.648068 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:31.648595 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:31.648611 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:31.651262 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":233,"threads_started":1}
W20260807 03:03:31.651727 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:31.643819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:34275 (local address 127.5.122.62:44187)
0807 03:03:31.644034 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:31.644040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:31.644301 (+ 261us) server_negotiation.cc:407] Connection header received
0807 03:03:31.644440 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:31.644443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:31.644504 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:31.644579 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:31.645240 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.646061 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.647335 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.647753 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.647857 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:31.651496 (+ 3639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:34275: BlockingRecv error: recv got EOF from 127.5.122.3:34275 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 03:03:31 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:31 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:31.786474 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:31.778214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:43125)
0807 03:03:31.778744 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:31.778759 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:31.778883 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:31.779178 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:31.779182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:31.779667 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:31.779826 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:31.779835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.780866 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.780871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:31.781766 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:31.781778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.782524 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.782530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:31.782699 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:31.783206 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:31.783229 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:31.786187 (+ 2958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":267,"threads_started":1}
W20260807 03:03:31.786608 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:31.778280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:43125 (local address 127.5.122.62:44187)
0807 03:03:31.778531 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:31.778538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:31.778835 (+ 297us) server_negotiation.cc:407] Connection header received
0807 03:03:31.778990 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:31.778996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:31.779058 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:31.779137 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:31.779969 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.780734 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.781910 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:31.782382 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:31.782487 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:31.786415 (+ 3928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:43125: BlockingRecv error: recv got EOF from 127.5.122.1:43125 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 03:03:32 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:32 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:32.204721 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:32.196782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:45909)
0807 03:03:32.197294 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:32.197310 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:32.197433 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:32.197790 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:32.197794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:32.198201 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:32.198418 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:32.198431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.199346 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.199350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:32.200252 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:32.200263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.200845 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.200849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:32.200971 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:32.201484 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:32.201500 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:32.204428 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":249,"threads_started":1}
W20260807 03:03:32.204934 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:32.196928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:45909 (local address 127.5.122.62:44187)
0807 03:03:32.197127 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:32.197133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:32.197377 (+ 244us) server_negotiation.cc:407] Connection header received
0807 03:03:32.197523 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:32.197527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:32.197587 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:32.197662 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:32.198532 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.199244 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.200353 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.200744 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.200827 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:32.204657 (+ 3830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:45909: BlockingRecv error: recv got EOF from 127.5.122.2:45909 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 03:03:32 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:32 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:32.661309 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:32.652457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:37423)
0807 03:03:32.652931 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:32.652948 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:32.653062 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:32.653415 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:32.653421 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:32.653985 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:32.654259 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:32.654278 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.655616 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.655622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:32.656572 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:32.656585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.657359 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.657365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:32.657571 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:32.658145 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:32.658165 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:32.661062 (+ 2897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":234,"threads_started":1}
W20260807 03:03:32.661564 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:32.652563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:37423 (local address 127.5.122.62:44187)
0807 03:03:32.652775 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:32.652780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:32.653020 (+ 240us) server_negotiation.cc:407] Connection header received
0807 03:03:32.653147 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:32.653151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:32.653213 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:32.653307 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:32.654475 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.655437 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.656724 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.657201 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.657329 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:32.661268 (+ 3939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:37423: BlockingRecv error: recv got EOF from 127.5.122.3:37423 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 03:03:32 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:32 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:32.795192 18993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:32.787319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:47795)
0807 03:03:32.787832 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:32.787847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:32.787968 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:32.788218 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:32.788221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:32.788599 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:32.788749 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:32.788759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.789675 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.789678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:32.790598 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:32.790609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.791271 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.791277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:32.791434 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:32.791963 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:32.791981 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:32.794875 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":289,"threads_started":1}
W20260807 03:03:32.795373 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:32.787364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:47795 (local address 127.5.122.62:44187)
0807 03:03:32.787544 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:32.787548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:32.787920 (+ 372us) server_negotiation.cc:407] Connection header received
0807 03:03:32.788052 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:32.788056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:32.788116 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:32.788192 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:32.788852 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.789576 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.790697 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:32.791111 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:32.791195 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:32.795091 (+ 3896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:47795: BlockingRecv error: recv got EOF from 127.5.122.1:47795 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 03:03:33 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:33 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:33.213583 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:33.205674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:48979)
0807 03:03:33.206172 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:33.206189 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:33.206346 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:33.206670 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:33.206675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:33.207073 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:33.207229 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:33.207239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.208237 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.208241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:33.209147 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:33.209160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.209902 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.209909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:33.210067 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:33.210604 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:33.210638 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:33.213354 (+ 2716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":278,"threads_started":1}
W20260807 03:03:33.213810 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:33.205730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:48979 (local address 127.5.122.62:44187)
0807 03:03:33.205916 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:33.205922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:33.206269 (+ 347us) server_negotiation.cc:407] Connection header received
0807 03:03:33.206471 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:33.206476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:33.206542 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:33.206623 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:33.207385 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.208102 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.209281 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.209751 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.209867 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:33.213563 (+ 3696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:48979: BlockingRecv error: recv got EOF from 127.5.122.2:48979 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 03:03:33 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:33 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:33.673230 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:33.662169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:54191)
0807 03:03:33.662665 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:33.662682 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:33.662796 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:33.663048 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:33.663051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:33.663414 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:33.663561 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:33.663569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.664556 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.664559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:33.665668 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:33.665685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.666483 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.666489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:33.666658 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:33.670394 (+ 3736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:33.670412 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:33.672999 (+ 2587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":275,"threads_started":1}
W20260807 03:03:33.673393 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:33.662262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:54191 (local address 127.5.122.62:44187)
0807 03:03:33.662482 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:33.662488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:33.662754 (+ 266us) server_negotiation.cc:407] Connection header received
0807 03:03:33.662886 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:33.662890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:33.662951 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:33.663018 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:33.663684 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.664456 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.665799 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.666305 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.666451 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:33.673164 (+ 6713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:54191: BlockingRecv error: recv got EOF from 127.5.122.3:54191 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 03:03:33 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:33 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:33.803884 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:33.796103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:34377)
0807 03:03:33.796630 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:33.796647 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:33.796771 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:33.797035 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:33.797038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:33.797462 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:33.797630 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:33.797640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.798605 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.798609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:33.799536 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:33.799547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.800137 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.800141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:33.800274 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:33.800803 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:33.800820 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:33.803576 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":234,"threads_started":1}
W20260807 03:03:33.804003 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:33.796137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:34377 (local address 127.5.122.62:44187)
0807 03:03:33.796333 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:33.796340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:33.796717 (+ 377us) server_negotiation.cc:407] Connection header received
0807 03:03:33.796862 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:33.796866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:33.796930 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:33.797003 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:33.797738 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.798498 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.799638 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:33.800029 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:33.800114 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:33.803752 (+ 3638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:34377: BlockingRecv error: recv got EOF from 127.5.122.1:34377 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 03:03:34 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:34 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 03:03:34.222746 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:34.214533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:36505 (local address 127.5.122.62:44187)
0807 03:03:34.214770 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:34.214778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:34.215135 (+ 357us) server_negotiation.cc:407] Connection header received
0807 03:03:34.215279 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:34.215283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:34.215362 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:34.215440 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:34.216174 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.216932 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.218091 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.218538 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.218630 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:34.222461 (+ 3831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:36505: BlockingRecv error: recv got EOF from 127.5.122.2:36505 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 03:03:34.223455 18997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:34.214501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:36505)
0807 03:03:34.215044 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:34.215062 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:34.215187 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:34.215471 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:34.215474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:34.215901 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:34.216066 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:34.216075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.217038 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.217041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:34.217975 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:34.218000 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.218655 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.218659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:34.218770 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:34.219332 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:34.219349 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:34.223146 (+ 3797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":252,"threads_started":1}
Aug 07 03:03:34 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:34 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:34.682113 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:34.674131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:52273)
0807 03:03:34.674799 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:34.674820 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:34.674971 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:34.675352 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:34.675355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:34.675900 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:34.676120 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:34.676131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.677073 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.677076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:34.677966 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:34.677977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.678663 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.678668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:34.678779 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:34.679293 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:34.679310 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:34.681877 (+ 2567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":333,"threads_started":1}
W20260807 03:03:34.682260 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:34.674199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:52273 (local address 127.5.122.62:44187)
0807 03:03:34.674388 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:34.674395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:34.674924 (+ 529us) server_negotiation.cc:407] Connection header received
0807 03:03:34.675106 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:34.675111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:34.675193 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:34.675288 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:34.676249 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.676965 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.678080 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.678534 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.678626 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:34.682061 (+ 3435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:52273: BlockingRecv error: recv got EOF from 127.5.122.3:52273 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 03:03:34 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:34 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:34.812433 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:34.804807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:40081)
0807 03:03:34.805288 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:34.805304 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:34.805420 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:34.805672 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:34.805675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:34.806075 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:34.806238 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:34.806247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.807190 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.807193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:34.808088 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:34.808098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.808733 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.808737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:34.808848 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:34.809389 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:34.809407 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:34.812103 (+ 2696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":232,"threads_started":1}
W20260807 03:03:34.812587 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:34.804969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:40081 (local address 127.5.122.62:44187)
0807 03:03:34.805199 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:34.805205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:34.805375 (+ 170us) server_negotiation.cc:407] Connection header received
0807 03:03:34.805504 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:34.805507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:34.805565 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:34.805644 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:34.806372 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.807080 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.808187 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:34.808595 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:34.808678 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:34.812308 (+ 3630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:40081: BlockingRecv error: recv got EOF from 127.5.122.1:40081 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 03:03:35 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:35 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:35.233428 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:35.224501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:58483)
0807 03:03:35.225086 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:35.225107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:35.225256 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:35.225661 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:35.225666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:35.226171 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:35.226404 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:35.226419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.227920 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.227927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:35.228842 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:35.228855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.229544 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.229548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:35.229657 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:35.230171 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:35.230188 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:35.233207 (+ 3019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":343,"threads_started":1}
W20260807 03:03:35.233666 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:35.224612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:58483 (local address 127.5.122.62:44187)
0807 03:03:35.224950 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:35.224983 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:35.225211 (+ 228us) server_negotiation.cc:407] Connection header received
0807 03:03:35.225419 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:35.225425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:35.225504 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:35.225599 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:35.226552 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.227646 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.228971 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.229409 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.229501 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:35.233434 (+ 3933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:58483: BlockingRecv error: recv got EOF from 127.5.122.2:58483 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Aug 07 03:03:35 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:35 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:35.691160 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:35.683012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:47247)
0807 03:03:35.683509 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:35.683524 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:35.683654 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:35.683971 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:35.683975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:35.684353 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:35.684501 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:35.684510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.685522 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.685526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:35.686454 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:35.686467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.687253 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.687260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:35.687411 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:35.687977 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:35.687997 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:35.690873 (+ 2876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":259,"threads_started":1}
W20260807 03:03:35.691231 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:35.683100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:47247 (local address 127.5.122.62:44187)
0807 03:03:35.683305 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:35.683312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:35.683607 (+ 295us) server_negotiation.cc:407] Connection header received
0807 03:03:35.683763 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:35.683768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:35.683830 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:35.683910 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:35.684642 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.685383 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.686587 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.687077 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.687196 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:35.691050 (+ 3854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:47247: BlockingRecv error: recv got EOF from 127.5.122.3:47247 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 03:03:35 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:35 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:35.821163 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:35.813248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:36829)
0807 03:03:35.813748 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:35.813766 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:35.813942 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:35.814348 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:35.814353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:35.814777 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:35.814950 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:35.814960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.816078 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.816083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:35.816997 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:35.817008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.817608 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.817612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:35.817731 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:35.818252 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:35.818269 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:35.820915 (+ 2646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":274,"threads_started":1}
W20260807 03:03:35.821305 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:35.813331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:36829 (local address 127.5.122.62:44187)
0807 03:03:35.813534 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:35.813540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:35.813857 (+ 317us) server_negotiation.cc:407] Connection header received
0807 03:03:35.814081 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:35.814086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:35.814168 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:35.814268 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:35.815087 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.815955 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.817101 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:35.817500 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:35.817586 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:35.821098 (+ 3512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:36829: BlockingRecv error: recv got EOF from 127.5.122.1:36829 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 03:03:36 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:36 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:36.246490 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:36.238264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:36657)
0807 03:03:36.238794 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:36.238810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:36.238938 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:36.239248 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:36.239253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:36.239677 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:36.239845 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:36.239854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.240852 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.240856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:36.241797 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:36.241810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.242546 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.242553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:36.242712 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:36.243200 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:36.243217 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:36.246197 (+ 2980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":258,"threads_started":1}
W20260807 03:03:36.246608 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:36.238359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:36657 (local address 127.5.122.62:44187)
0807 03:03:36.238545 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:36.238551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:36.238895 (+ 344us) server_negotiation.cc:407] Connection header received
0807 03:03:36.239054 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:36.239060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:36.239128 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:36.239202 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:36.239982 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.240710 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.241934 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.242390 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.242496 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:36.246411 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:36657: BlockingRecv error: recv got EOF from 127.5.122.2:36657 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 03:03:36 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:36 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:36.702775 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:36.692080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:33095)
0807 03:03:36.692607 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:36.692624 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:36.692753 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:36.693033 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:36.693038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:36.693427 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:36.693598 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:36.693607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.694674 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.694678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:36.695749 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:36.695765 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.696440 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.696444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:36.696570 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:36.697222 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:36.697240 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:36.702437 (+ 5197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":281,"threads_started":1}
W20260807 03:03:36.702870 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:36.692166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:33095 (local address 127.5.122.62:44187)
0807 03:03:36.692323 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:36.692328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:36.692690 (+ 362us) server_negotiation.cc:407] Connection header received
0807 03:03:36.692843 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:36.692847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:36.692909 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:36.692986 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:36.693710 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.694552 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.695864 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.696315 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.696415 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:36.702654 (+ 6239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:33095: BlockingRecv error: recv got EOF from 127.5.122.3:33095 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 03:03:36 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:36 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:36.830236 19006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:36.822055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:53279)
0807 03:03:36.822596 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:36.822612 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:36.822736 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:36.823054 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:36.823058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:36.823445 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:36.823600 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:36.823610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.824664 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.824670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:36.825778 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:36.825790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.826567 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.826574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:36.826718 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:36.827237 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:36.827254 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:36.829913 (+ 2659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":311,"threads_started":1}
W20260807 03:03:36.830379 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:36.822151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:53279 (local address 127.5.122.62:44187)
0807 03:03:36.822392 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:36.822399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:36.822694 (+ 295us) server_negotiation.cc:407] Connection header received
0807 03:03:36.822844 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:36.822849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:36.822924 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:36.823004 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:36.823739 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.824543 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.825911 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:36.826414 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:36.826517 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:36.830120 (+ 3603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:53279: BlockingRecv error: recv got EOF from 127.5.122.1:53279 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 03:03:37 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:37 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:37.255541 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:37.247409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:55777)
0807 03:03:37.247955 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:37.247973 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:37.248097 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:37.248406 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:37.248410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:37.248782 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:37.248944 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:37.248953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.249827 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.249830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:37.250932 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:37.250947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.251828 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.251835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:37.252004 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:37.252575 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:37.252592 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:37.255215 (+ 2623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":288,"threads_started":1}
W20260807 03:03:37.255682 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:37.247465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:55777 (local address 127.5.122.62:44187)
0807 03:03:37.247635 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:37.247639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:37.248039 (+ 400us) server_negotiation.cc:407] Connection header received
0807 03:03:37.248186 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:37.248190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:37.248248 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:37.248321 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:37.249056 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.249733 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.251067 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.251651 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.251776 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:37.255428 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:55777: BlockingRecv error: recv got EOF from 127.5.122.2:55777 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 03:03:37 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:37 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:37.711443 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:37.703659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:50079)
0807 03:03:37.704156 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:37.704173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:37.704306 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:37.704629 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:37.704635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:37.705030 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:37.705174 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:37.705183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.706151 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.706156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:37.707148 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:37.707160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.707819 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.707824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:37.707983 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:37.708504 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:37.708521 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:37.711116 (+ 2595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":260,"threads_started":1}
W20260807 03:03:37.711611 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:37.703808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:50079 (local address 127.5.122.62:44187)
0807 03:03:37.704063 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:37.704069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:37.704248 (+ 179us) server_negotiation.cc:407] Connection header received
0807 03:03:37.704434 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:37.704438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:37.704499 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:37.704576 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:37.705314 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.706019 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.707276 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.707674 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.707782 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:37.711339 (+ 3557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:50079: BlockingRecv error: recv got EOF from 127.5.122.3:50079 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 07 03:03:37 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:37 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:37.839052 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:37.831184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:43287)
0807 03:03:37.831699 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:37.831718 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:37.831852 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:37.832223 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:37.832228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:37.832627 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:37.832807 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:37.832816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.833727 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.833730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:37.834646 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:37.834657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.835266 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.835270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:37.835380 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:37.835864 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:37.835887 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:37.838691 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":264,"threads_started":1}
W20260807 03:03:37.839120 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:37.831227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:43287 (local address 127.5.122.62:44187)
0807 03:03:37.831441 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:37.831447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:37.831800 (+ 353us) server_negotiation.cc:407] Connection header received
0807 03:03:37.831975 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:37.831978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:37.832064 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:37.832149 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:37.832917 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.833628 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.834751 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:37.835161 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:37.835248 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:37.838899 (+ 3651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:43287: BlockingRecv error: recv got EOF from 127.5.122.1:43287 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 03:03:38 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:38 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:38.264097 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:38.256419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:35787)
0807 03:03:38.256873 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:38.256891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:38.257001 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:38.257263 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:38.257265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:38.257670 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:38.257848 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:38.257856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.258847 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.258851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:38.259741 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:38.259753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.260356 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.260359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:38.260489 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:38.261016 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:38.261032 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:38.263694 (+ 2662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":243,"threads_started":1}
W20260807 03:03:38.264137 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:38.256488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:35787 (local address 127.5.122.62:44187)
0807 03:03:38.256678 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:38.256686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:38.256960 (+ 274us) server_negotiation.cc:407] Connection header received
0807 03:03:38.257102 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:38.257105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:38.257167 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:38.257233 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:38.257953 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.258746 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.259844 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.260251 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.260344 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:38.263916 (+ 3572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:35787: BlockingRecv error: recv got EOF from 127.5.122.2:35787 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 03:03:38 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:38 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:38.723024 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:38.712328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:33813)
0807 03:03:38.712847 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:38.712863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:38.712989 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:38.713291 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:38.713296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:38.713696 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:38.713878 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:38.713887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.714924 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.714928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:38.715898 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:38.715910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.716675 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.716680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:38.716857 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:38.720181 (+ 3324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:38.720199 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:38.722769 (+ 2570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":293,"threads_started":1}
W20260807 03:03:38.723130 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:38.712447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:33813 (local address 127.5.122.62:44187)
0807 03:03:38.712629 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:38.712635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:38.712945 (+ 310us) server_negotiation.cc:407] Connection header received
0807 03:03:38.713100 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:38.713104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:38.713164 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:38.713240 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:38.714017 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.714797 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.716016 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.716536 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.716632 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:38.722947 (+ 6315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:33813: BlockingRecv error: recv got EOF from 127.5.122.3:33813 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 03:03:38 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:38 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:38.848129 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:38.839997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:55693)
0807 03:03:38.840500 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:38.840516 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:38.840640 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:38.840964 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:38.840969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:38.841422 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:38.841582 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:38.841592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.842827 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.842832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:38.843861 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:38.843883 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.844596 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.844602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:38.844766 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:38.845283 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:38.845300 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:38.847789 (+ 2489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":272,"threads_started":1}
W20260807 03:03:38.848223 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:38.840032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:55693 (local address 127.5.122.62:44187)
0807 03:03:38.840217 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:38.840224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:38.840596 (+ 372us) server_negotiation.cc:407] Connection header received
0807 03:03:38.840750 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:38.840755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:38.840823 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:38.840901 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:38.841729 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.842693 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.844007 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:38.844462 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:38.844568 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:38.848001 (+ 3433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:55693: BlockingRecv error: recv got EOF from 127.5.122.1:55693 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 03:03:39 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:39 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:39.272905 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:39.264964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:56541)
0807 03:03:39.265440 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:39.265456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:39.265567 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:39.265830 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:39.265833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:39.266204 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:39.266386 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:39.266396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.267368 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.267371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:39.268272 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:39.268283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.268923 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.268926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:39.269043 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:39.269597 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:39.269614 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:39.272611 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":243,"threads_started":1}
W20260807 03:03:39.272977 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:39.264988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:56541 (local address 127.5.122.62:44187)
0807 03:03:39.265191 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:39.265199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:39.265528 (+ 329us) server_negotiation.cc:407] Connection header received
0807 03:03:39.265652 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:39.265655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:39.265716 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:39.265789 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:39.266508 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.267244 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.268381 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.268813 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.268900 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:39.272794 (+ 3894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:56541: BlockingRecv error: recv got EOF from 127.5.122.2:56541 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 03:03:39 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:39 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:39.731356 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:39.723933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:54145)
0807 03:03:39.724457 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:39.724474 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:39.724587 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:39.724851 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:39.724854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:39.725211 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:39.725369 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:39.725377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.726337 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.726340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:39.727247 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:39.727258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.727854 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.727857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:39.727985 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:39.728520 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:39.728537 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:39.731126 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":270,"threads_started":1}
W20260807 03:03:39.731531 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:39.723997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:54145 (local address 127.5.122.62:44187)
0807 03:03:39.724160 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:39.724165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:39.724537 (+ 372us) server_negotiation.cc:407] Connection header received
0807 03:03:39.724670 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:39.724674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:39.724733 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:39.724807 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:39.725484 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.726200 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.727345 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.727751 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.727839 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:39.731315 (+ 3476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:54145: BlockingRecv error: recv got EOF from 127.5.122.3:54145 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 03:03:39 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:39 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:39.856954 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:39.849001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:46575)
0807 03:03:39.849527 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:39.849542 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:39.849670 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:39.849999 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:39.850004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:39.850440 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:39.850604 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:39.850612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.851583 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.851589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:39.852551 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:39.852565 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.853333 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.853338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:39.853509 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:39.854067 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:39.854085 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:39.856636 (+ 2551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":261,"threads_started":1}
W20260807 03:03:39.857120 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:39.849133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:46575 (local address 127.5.122.62:44187)
0807 03:03:39.849362 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:39.849368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:39.849620 (+ 252us) server_negotiation.cc:407] Connection header received
0807 03:03:39.849795 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:39.849799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:39.849858 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:39.849944 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:39.850740 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.851449 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.852701 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:39.853159 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:39.853267 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:39.856840 (+ 3573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:46575: BlockingRecv error: recv got EOF from 127.5.122.1:46575 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 03:03:40 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:40 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:40.282020 19020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:40.273832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:47401)
0807 03:03:40.274407 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:40.274424 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:40.274555 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:40.274877 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:40.274882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:40.275334 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:40.275508 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:40.275520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.276753 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.276757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:40.277666 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:40.277678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.278357 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.278363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:40.278497 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:40.279014 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:40.279031 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:40.281700 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":259,"threads_started":1}
W20260807 03:03:40.282155 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:40.274012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:47401 (local address 127.5.122.62:44187)
0807 03:03:40.274200 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:40.274206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:40.274499 (+ 293us) server_negotiation.cc:407] Connection header received
0807 03:03:40.274665 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:40.274671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:40.274741 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:40.274821 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:40.275651 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.276636 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.277773 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.278199 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.278286 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:40.281925 (+ 3639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:47401: BlockingRecv error: recv got EOF from 127.5.122.2:47401 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 03:03:40 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:40 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:40.740244 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:40.732344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:40303)
0807 03:03:40.732813 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:40.732830 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:40.732953 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:40.733214 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:40.733217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:40.733594 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:40.733761 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:40.733770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.734819 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.734822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:40.735734 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:40.735747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.736403 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.736407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:40.736517 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:40.737055 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:40.737077 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:40.739983 (+ 2906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":229,"threads_started":1}
W20260807 03:03:40.740356 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:40.732412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:40303 (local address 127.5.122.62:44187)
0807 03:03:40.732610 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:40.732616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:40.732908 (+ 292us) server_negotiation.cc:407] Connection header received
0807 03:03:40.733043 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:40.733047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:40.733109 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:40.733184 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:40.733869 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.734720 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.735854 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.736262 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.736357 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:40.740157 (+ 3800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:40303: BlockingRecv error: recv got EOF from 127.5.122.3:40303 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 03:03:40 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:40 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:40.865422 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:40.857807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:39843)
0807 03:03:40.858291 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:40.858306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:40.858466 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:40.858764 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:40.858769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:40.859162 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:40.859331 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:40.859340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.860344 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.860350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:40.861259 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:40.861270 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.861942 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.861947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:40.862113 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:40.862648 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:40.862667 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:40.865184 (+ 2517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":248,"threads_started":1}
W20260807 03:03:40.865571 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:40.857890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:39843 (local address 127.5.122.62:44187)
0807 03:03:40.858184 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:40.858190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:40.858406 (+ 216us) server_negotiation.cc:407] Connection header received
0807 03:03:40.858576 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:40.858581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:40.858642 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:40.858718 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:40.859471 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.860206 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.861396 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:40.861799 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:40.861905 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:40.865368 (+ 3463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:39843: BlockingRecv error: recv got EOF from 127.5.122.1:39843 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
Aug 07 03:03:41 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:41 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:41.290660 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:41.282775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:51753)
0807 03:03:41.283209 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:41.283224 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:41.283354 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:41.283703 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:41.283709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:41.284145 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:41.284371 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:41.284402 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.285633 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.285637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:41.286582 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:41.286596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.287219 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.287223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:41.287332 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:41.287852 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:41.287868 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:41.290407 (+ 2539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":220,"threads_started":1}
W20260807 03:03:41.290828 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:41.282854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:51753 (local address 127.5.122.62:44187)
0807 03:03:41.283053 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:41.283059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:41.283302 (+ 243us) server_negotiation.cc:407] Connection header received
0807 03:03:41.283485 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:41.283490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:41.283553 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:41.283640 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:41.284612 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.285502 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.286697 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.287115 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.287204 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:41.290601 (+ 3397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:51753: BlockingRecv error: recv got EOF from 127.5.122.2:51753 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 03:03:41 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:41 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.3@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:41.748342 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:41.741072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.3:51379)
0807 03:03:41.741485 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:41.741503 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:41.741615 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:41.741886 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:41.741889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:41.742244 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:41.742463 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:41.742472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.743381 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.743385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:41.744254 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:41.744264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.744869 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.744873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:41.744989 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:41.745553 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:41.745570 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:41.748110 (+ 2540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":205,"threads_started":1}
W20260807 03:03:41.748529 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:41.741140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:51379 (local address 127.5.122.62:44187)
0807 03:03:41.741347 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:41.741353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:41.741573 (+ 220us) server_negotiation.cc:407] Connection header received
0807 03:03:41.741700 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:41.741704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:41.741780 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:41.741855 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:41.742570 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.743261 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.744359 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.744766 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.744853 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:41.748320 (+ 3467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.3:51379: BlockingRecv error: recv got EOF from 127.5.122.3:51379 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 03:03:41 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:41 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.1@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:41.874747 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:41.866581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.1:60563)
0807 03:03:41.867117 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:41.867134 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:41.867268 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:41.867621 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:41.867626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:41.868075 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:41.868268 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:41.868278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.869343 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.869347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:41.870233 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:41.870245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.870961 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.870967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:41.871128 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:41.871718 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:41.871736 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:41.874487 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":299,"threads_started":1}
W20260807 03:03:41.874878 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:41.866684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:60563 (local address 127.5.122.62:44187)
0807 03:03:41.866909 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:41.866915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:41.867219 (+ 304us) server_negotiation.cc:407] Connection header received
0807 03:03:41.867402 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:41.867408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:41.867474 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:41.867557 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:41.868433 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.869199 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.870385 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:41.870820 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:41.870936 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:41.874669 (+ 3733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.1:60563: BlockingRecv error: recv got EOF from 127.5.122.1:60563 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 03:03:42 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 03:03:42 dist-test-slave-fgb3 krb5kdc[17428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.2@KRBTEST.COM for oryx/127.5.122.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 03:03:42.299948 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:42.291855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:44187 (local address 127.5.122.2:37247)
0807 03:03:42.292415 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:42.292434 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:42.292554 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:42.292825 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:42.292827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:42.293219 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:42.293458 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:42.293474 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.294562 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.294569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:42.295767 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:42.295783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.296471 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.296476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:42.296597 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:42.297131 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:42.297147 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:42.299689 (+ 2542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.62:44187: Server oryx/127.5.122.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":254,"threads_started":1}
W20260807 03:03:42.300063 18572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:42.291942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:37247 (local address 127.5.122.62:44187)
0807 03:03:42.292147 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:42.292154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:42.292495 (+ 341us) server_negotiation.cc:407] Connection header received
0807 03:03:42.292649 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:42.292652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:42.292714 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:42.292785 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:42.293593 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.294422 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.295885 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.296323 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.296415 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:42.299861 (+ 3446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.2:37247: BlockingRecv error: recv got EOF from 127.5.122.2:37247 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 03:03:42.579926 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 18349
I20260807 03:03:42.591658 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:42.581065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.5.122.1:32933)
0807 03:03:42.581653 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:42.581661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:42.581688 (+ 27us) server_negotiation.cc:407] Connection header received
0807 03:03:42.581748 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:42.581754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:42.582000 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:42.582157 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:42.583525 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.584646 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.585765 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.586373 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.586466 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:42.588777 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:42.588786 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:42.588807 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:42.588846 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:42.591174 (+ 2328us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.122.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 03:03:42.591434 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41440: Request ticket server kudu/127.5.122.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":292,"threads_started":1}
W20260807 03:03:42.591773 19031 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41440: Request ticket server kudu/127.5.122.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 03:03:42.591864 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 18349: Not authorized: Client connection negotiation failed: client connection to 127.5.122.1:32933: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.122.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 03:03:42.592094 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 18349
I20260807 03:03:42.613430 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 18575
I20260807 03:03:42.625589 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:42.614514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.5.122.2:33011)
0807 03:03:42.615123 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:42.615131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:42.615162 (+ 31us) server_negotiation.cc:407] Connection header received
0807 03:03:42.615227 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:42.615233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:42.615479 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:42.615665 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:42.617005 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.618083 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.619398 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.619962 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.620081 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:42.622579 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:42.622588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:42.622603 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:42.622641 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:42.624899 (+ 2258us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.122.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 03:03:42.625231 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33382: Request ticket server kudu/127.5.122.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":289,"threads_started":1}
W20260807 03:03:42.625696 19032 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33382: Request ticket server kudu/127.5.122.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 03:03:42.625680 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 18575: Not authorized: Client connection negotiation failed: client connection to 127.5.122.2:33011: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.122.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 03:03:42.625835 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 18575
I20260807 03:03:42.648052 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 18783
I20260807 03:03:42.659180 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:42.648986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.5.122.3:36375)
0807 03:03:42.649575 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:42.649582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:42.649609 (+ 27us) server_negotiation.cc:407] Connection header received
0807 03:03:42.649676 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:42.649682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:42.649919 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:42.650081 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:42.651346 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.652417 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.653518 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:42.654063 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:42.654154 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:42.656421 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:42.656429 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:42.656442 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:42.656478 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:42.658706 (+ 2228us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.122.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 03:03:42.658948 (+ 242us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37408: Request ticket server kudu/127.5.122.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":312,"threads_started":1}
W20260807 03:03:42.659281 19033 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37408: Request ticket server kudu/127.5.122.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 03:03:42.659312 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 18783: Not authorized: Client connection negotiation failed: client connection to 127.5.122.3:36375: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.122.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 03:03:42.659485 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 18783
I20260807 03:03:42.680346 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 18233
I20260807 03:03:42.681653 18337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:42.795986 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 18233
2026-08-07T03:03:42Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21052 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T03:03:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:42Z Disabled control of system clock
I20260807 03:03:42.880905 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:37905
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:37905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 03:03:43.199087 19043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:43.199359 19043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:43.199489 19043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:43.206489 19043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:43.206591 19043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:43.206633 19043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:43.206691 19043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:43.206733 19043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:43.215538 19043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:37905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:37905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:43.216745 19043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:43.218446 19043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:43.229249 19053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:43.229341 19054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:43.229461 19057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:43.230075 19043 server_base.cc:1034] running on GCE node
I20260807 03:03:43.230751 19043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:43.231940 19043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:43.233122 19043 hybrid_clock.cc:648] HybridClock initialized: now 1786071823233077 us; error 40 us; skew 500 ppm
I20260807 03:03:43.233429 19043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:43.235489 19043 webserver.cc:460] Webserver started at http://127.5.122.62:35265/ using document root <none> and password file <none>
I20260807 03:03:43.236104 19043 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:43.236197 19043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:43.236513 19043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:43.239243 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "03f42ea69c4b42deba4fd818eefd9542"
format_stamp: "Formatted at 2026-08-07 03:03:43 on dist-test-slave-fgb3"
server_key: "a01c7f685b0190df74d68e4bec6d351f"
server_key_iv: "8f2ba6e05c673c654eaa75f45fcb9b51"
server_key_version: "encryptionkey@0"
I20260807 03:03:43.239996 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "03f42ea69c4b42deba4fd818eefd9542"
format_stamp: "Formatted at 2026-08-07 03:03:43 on dist-test-slave-fgb3"
server_key: "a01c7f685b0190df74d68e4bec6d351f"
server_key_iv: "8f2ba6e05c673c654eaa75f45fcb9b51"
server_key_version: "encryptionkey@0"
I20260807 03:03:43.244697 19043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:03:43.247869 19066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:43.249046 19043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 03:03:43.249191 19043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "03f42ea69c4b42deba4fd818eefd9542"
format_stamp: "Formatted at 2026-08-07 03:03:43 on dist-test-slave-fgb3"
server_key: "a01c7f685b0190df74d68e4bec6d351f"
server_key_iv: "8f2ba6e05c673c654eaa75f45fcb9b51"
server_key_version: "encryptionkey@0"
I20260807 03:03:43.249393 19043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:43.275851 19043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:43.276732 19043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:43.277026 19043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:43.294463 19043 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:37905
I20260807 03:03:43.294497 19144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:37905 every 8 connection(s)
I20260807 03:03:43.295720 19043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:43.299379 19146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:43.301426 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 19043
I20260807 03:03:43.301625 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:03:43.302500 5608 external_mini_cluster.cc:1428] Setting key 8a365542712bbaf55efca461c6471f35
I20260807 03:03:43.311416 19146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542: Bootstrap starting.
I20260807 03:03:43.315018 19146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:43.316376 19146 log.cc:824] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:43.317608 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:43.304628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.5.122.62:37905)
0807 03:03:43.305596 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:43.305606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:43.305646 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:03:43.307272 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:43.307294 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:43.307710 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:43.308032 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:43.308637 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:43.309896 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:43.311706 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:43.312780 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:43.312989 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:43.314371 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:43.314383 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:43.314401 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:43.315242 (+ 841us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:43.315402 (+ 160us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:43.315431 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:43.315554 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:43.316124 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 03:03:43.316450 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":369,"threads_started":1}
I20260807 03:03:43.319648 19146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542: No bootstrap required, opened a new log
I20260807 03:03:43.326778 19146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f42ea69c4b42deba4fd818eefd9542" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37905 } }
I20260807 03:03:43.327144 19146 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:43.327243 19146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f42ea69c4b42deba4fd818eefd9542, State: Initialized, Role: FOLLOWER
I20260807 03:03:43.327780 19146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f42ea69c4b42deba4fd818eefd9542" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37905 } }
I20260807 03:03:43.327994 19146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:43.328120 19146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:43.328264 19146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:43.330250 19146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f42ea69c4b42deba4fd818eefd9542" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37905 } }
I20260807 03:03:43.330749 19146 leader_election.cc:302] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03f42ea69c4b42deba4fd818eefd9542; no voters:
I20260807 03:03:43.331264 19146 leader_election.cc:288] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:03:43.331423 19154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:43.333066 19146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:43.338248 19154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [term 1 LEADER]: Becoming Leader. State: Replica: 03f42ea69c4b42deba4fd818eefd9542, State: Running, Role: LEADER
I20260807 03:03:43.338708 19154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f42ea69c4b42deba4fd818eefd9542" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37905 } }
I20260807 03:03:43.341511 19155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03f42ea69c4b42deba4fd818eefd9542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f42ea69c4b42deba4fd818eefd9542" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37905 } } }
I20260807 03:03:43.341687 19156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03f42ea69c4b42deba4fd818eefd9542. Latest consensus state: current_term: 1 leader_uuid: "03f42ea69c4b42deba4fd818eefd9542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f42ea69c4b42deba4fd818eefd9542" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37905 } } }
I20260807 03:03:43.341922 19155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:43.341977 19156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542 [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:43.343051 19177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:43.347241 19177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:43.354614 19177 catalog_manager.cc:1347] Generated new cluster ID: 52f6c6793b4c4fe2a6e19dfc72b854da
I20260807 03:03:43.354708 19177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:43.372998 19177 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:03:43.374513 19177 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:43.383070 19177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03f42ea69c4b42deba4fd818eefd9542: Generated new TSK 0
I20260807 03:03:43.383786 19177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:03:43.389846 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37905
--builtin_ntp_servers=127.5.122.20:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 03:03:43.710245 19184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:43.710563 19184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:43.710692 19184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:43.717828 19184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:43.717923 19184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:43.718094 19184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:43.727027 19184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:43.728225 19184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:43.729962 19184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:43.742300 19198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:43.742465 19194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:43.742436 19195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:43.743422 19184 server_base.cc:1034] running on GCE node
I20260807 03:03:43.744026 19184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:43.744851 19184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:43.746042 19184 hybrid_clock.cc:648] HybridClock initialized: now 1786071823745993 us; error 42 us; skew 500 ppm
I20260807 03:03:43.746332 19184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:43.754873 19184 webserver.cc:460] Webserver started at http://127.5.122.1:37327/ using document root <none> and password file <none>
I20260807 03:03:43.755517 19184 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:43.755611 19184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:43.755910 19184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:43.758843 19184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "3488767cd8b6456f8942c4821aa2a088"
format_stamp: "Formatted at 2026-08-07 03:03:43 on dist-test-slave-fgb3"
server_key: "4dde854ff1f9c5b845abc9e481b7d312"
server_key_iv: "73a3c4fd5086d0a3393f475734b586d5"
server_key_version: "encryptionkey@0"
I20260807 03:03:43.759639 19184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "3488767cd8b6456f8942c4821aa2a088"
format_stamp: "Formatted at 2026-08-07 03:03:43 on dist-test-slave-fgb3"
server_key: "4dde854ff1f9c5b845abc9e481b7d312"
server_key_iv: "73a3c4fd5086d0a3393f475734b586d5"
server_key_version: "encryptionkey@0"
I20260807 03:03:43.764635 19184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:03:43.767932 19208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:43.769407 19184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 03:03:43.769557 19184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "3488767cd8b6456f8942c4821aa2a088"
format_stamp: "Formatted at 2026-08-07 03:03:43 on dist-test-slave-fgb3"
server_key: "4dde854ff1f9c5b845abc9e481b7d312"
server_key_iv: "73a3c4fd5086d0a3393f475734b586d5"
server_key_version: "encryptionkey@0"
I20260807 03:03:43.769769 19184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:43.782967 19184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:43.783723 19184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:43.783993 19184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:43.784878 19184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:43.786262 19184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:43.786412 19184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:43.786521 19184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:43.786605 19184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:43.818215 19184 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:33705
I20260807 03:03:43.818250 19387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:33705 every 8 connection(s)
I20260807 03:03:43.819634 19184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:43.825110 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 19184
I20260807 03:03:43.825295 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:03:43.826088 5608 external_mini_cluster.cc:1428] Setting key 67f4af65dbd3ef926f81e3ceab9df938
I20260807 03:03:43.829339 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37905
--builtin_ntp_servers=127.5.122.20:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 03:03:43.833035 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:43.821995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:33117 (local address 127.5.122.62:37905)
0807 03:03:43.822435 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:43.822439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:43.823606 (+ 1167us) server_negotiation.cc:407] Connection header received
0807 03:03:43.825591 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:43.825596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:43.825670 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:43.825780 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:43.827096 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:43.828209 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:43.830395 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:43.830808 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:43.830915 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:43.832341 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:43.832345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:43.832349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:43.832365 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:43.832406 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:43.832419 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:43.832485 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:43.832709 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 03:03:43.832782 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":210,"threads_started":1}
I20260807 03:03:43.834268 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:43.822558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37905 (local address 127.5.122.1:33117)
0807 03:03:43.823456 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:43.823497 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:43.825280 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:43.826033 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:43.826045 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:43.826158 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:43.826914 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:43.826940 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:43.828400 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:43.828406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:43.830205 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:43.830221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:43.831028 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:43.831035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:43.831301 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:43.832018 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:03:43.832022 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:43.832072 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:03:43.832165 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:43.832500 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:43.832511 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:43.832519 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 03:03:43.832692 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 03:03:43.833050 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":298,"threads_started":1}
I20260807 03:03:43.835784 19389 heartbeater.cc:347] Connected to a master server at 127.5.122.62:37905
I20260807 03:03:43.836138 19389 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:43.837033 19389 heartbeater.cc:511] Master 127.5.122.62:37905 requested a full tablet report, sending...
I20260807 03:03:43.839112 19092 ts_manager.cc:194] Registered new tserver with Master: 3488767cd8b6456f8942c4821aa2a088 (127.5.122.1:33705)
I20260807 03:03:43.840771 19092 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.1:33117
W20260807 03:03:44.148481 19396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:44.148722 19396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:44.148811 19396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:44.155640 19396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:44.155721 19396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:44.155822 19396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:44.164331 19396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:44.165436 19396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:44.167109 19396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:44.179072 19402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:44.179116 19401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:44.179145 19404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:44.180092 19396 server_base.cc:1034] running on GCE node
I20260807 03:03:44.180663 19396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:44.181537 19396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:44.182722 19396 hybrid_clock.cc:648] HybridClock initialized: now 1786071824182672 us; error 42 us; skew 500 ppm
I20260807 03:03:44.182998 19396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:44.185074 19396 webserver.cc:460] Webserver started at http://127.5.122.2:32967/ using document root <none> and password file <none>
I20260807 03:03:44.185702 19396 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:44.185784 19396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:44.186051 19396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:44.188737 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "bb178a92ceea4b9db1fbed3e40497c89"
format_stamp: "Formatted at 2026-08-07 03:03:44 on dist-test-slave-fgb3"
server_key: "a75a0ea92f46c1c8fe8592741b96b7ad"
server_key_iv: "6b7227e2b0ab0fdd9d7bb6c5be3c6a3e"
server_key_version: "encryptionkey@0"
I20260807 03:03:44.189455 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "bb178a92ceea4b9db1fbed3e40497c89"
format_stamp: "Formatted at 2026-08-07 03:03:44 on dist-test-slave-fgb3"
server_key: "a75a0ea92f46c1c8fe8592741b96b7ad"
server_key_iv: "6b7227e2b0ab0fdd9d7bb6c5be3c6a3e"
server_key_version: "encryptionkey@0"
I20260807 03:03:44.194146 19396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260807 03:03:44.197268 19413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:44.198537 19396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:03:44.198671 19396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "bb178a92ceea4b9db1fbed3e40497c89"
format_stamp: "Formatted at 2026-08-07 03:03:44 on dist-test-slave-fgb3"
server_key: "a75a0ea92f46c1c8fe8592741b96b7ad"
server_key_iv: "6b7227e2b0ab0fdd9d7bb6c5be3c6a3e"
server_key_version: "encryptionkey@0"
I20260807 03:03:44.198835 19396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:44.212229 19396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:44.212984 19396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:44.213209 19396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:44.214114 19396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:44.215515 19396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:44.215602 19396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:44.215683 19396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:44.215740 19396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:44.246037 19396 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:40099
I20260807 03:03:44.246088 19578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:40099 every 8 connection(s)
I20260807 03:03:44.247314 19396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:44.254828 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 19396
I20260807 03:03:44.254987 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:03:44.255427 5608 external_mini_cluster.cc:1428] Setting key 8d702483056cebe2d4afb85e31bc9d87
I20260807 03:03:44.258296 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37905
--builtin_ntp_servers=127.5.122.20:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 03:03:44.260502 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:44.249589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:32919 (local address 127.5.122.62:37905)
0807 03:03:44.249803 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:44.249810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:44.251126 (+ 1316us) server_negotiation.cc:407] Connection header received
0807 03:03:44.252764 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:44.252769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:44.252834 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:44.252934 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:44.253884 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.254722 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.256390 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.256969 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.257078 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:44.259348 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:44.259353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:44.259358 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:44.259391 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:44.259430 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:44.259445 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:44.259499 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:44.259860 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 03:03:44.259987 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 03:03:44.261302 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:44.250084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37905 (local address 127.5.122.2:32919)
0807 03:03:44.250981 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:44.251024 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:44.252547 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:44.253124 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:44.253132 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:44.253211 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:44.253742 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:44.253770 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.254875 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.254879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:44.256254 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:44.256269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.257181 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.257189 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:44.257500 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:44.259009 (+ 1509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:03:44.259012 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:44.259060 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:03:44.259145 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:44.259581 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:44.259592 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:44.259598 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 03:03:44.259781 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 03:03:44.260136 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":302,"threads_started":1}
I20260807 03:03:44.262585 19579 heartbeater.cc:347] Connected to a master server at 127.5.122.62:37905
I20260807 03:03:44.262846 19579 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:44.263551 19579 heartbeater.cc:511] Master 127.5.122.62:37905 requested a full tablet report, sending...
I20260807 03:03:44.264815 19092 ts_manager.cc:194] Registered new tserver with Master: bb178a92ceea4b9db1fbed3e40497c89 (127.5.122.2:40099)
I20260807 03:03:44.265784 19092 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.2:32919
W20260807 03:03:44.577421 19584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:44.577670 19584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:44.577760 19584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:44.584820 19584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:44.584918 19584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:44.585027 19584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:44.594174 19584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:37905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:44.595345 19584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:44.597033 19584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:44.609619 19592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:44.609572 19593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:44.609551 19595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:44.610523 19584 server_base.cc:1034] running on GCE node
I20260807 03:03:44.611052 19584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:44.611887 19584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:44.613070 19584 hybrid_clock.cc:648] HybridClock initialized: now 1786071824613016 us; error 49 us; skew 500 ppm
I20260807 03:03:44.613336 19584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:44.615419 19584 webserver.cc:460] Webserver started at http://127.5.122.3:46757/ using document root <none> and password file <none>
I20260807 03:03:44.616036 19584 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:44.616119 19584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:44.616382 19584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:44.619091 19584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "0b7edb7a2d444fe8b80f308a9f2aef46"
format_stamp: "Formatted at 2026-08-07 03:03:44 on dist-test-slave-fgb3"
server_key: "7daa7ca3d6d810048b346d3f4cd528a0"
server_key_iv: "5ad08f359c4c022964cc1f1ef7df3657"
server_key_version: "encryptionkey@0"
I20260807 03:03:44.619848 19584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "0b7edb7a2d444fe8b80f308a9f2aef46"
format_stamp: "Formatted at 2026-08-07 03:03:44 on dist-test-slave-fgb3"
server_key: "7daa7ca3d6d810048b346d3f4cd528a0"
server_key_iv: "5ad08f359c4c022964cc1f1ef7df3657"
server_key_version: "encryptionkey@0"
I20260807 03:03:44.624512 19584 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:03:44.627681 19607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:44.628841 19584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 03:03:44.628975 19584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "0b7edb7a2d444fe8b80f308a9f2aef46"
format_stamp: "Formatted at 2026-08-07 03:03:44 on dist-test-slave-fgb3"
server_key: "7daa7ca3d6d810048b346d3f4cd528a0"
server_key_iv: "5ad08f359c4c022964cc1f1ef7df3657"
server_key_version: "encryptionkey@0"
I20260807 03:03:44.629137 19584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:44.645267 19584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:44.645941 19584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:44.646157 19584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:44.647032 19584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:44.648288 19584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:44.648372 19584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:44.648450 19584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:44.648504 19584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:44.678921 19584 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:41427
I20260807 03:03:44.679003 19774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:41427 every 8 connection(s)
I20260807 03:03:44.680178 19584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:44.683923 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 19584
I20260807 03:03:44.684087 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:03:44.684549 5608 external_mini_cluster.cc:1428] Setting key 57805689fcf23a2ea11e471566ff028a
I20260807 03:03:44.691987 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:44.682554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:35597 (local address 127.5.122.62:37905)
0807 03:03:44.682715 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:44.682719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:44.684153 (+ 1434us) server_negotiation.cc:407] Connection header received
0807 03:03:44.686046 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:44.686051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:44.686129 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:44.686214 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:44.687278 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.687993 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.689462 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.689864 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.689973 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:44.691195 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:44.691201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:44.691205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:44.691222 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:44.691257 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:44.691271 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:44.691347 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:44.691571 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 03:03:44.691664 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:03:44.692881 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:44.683003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37905 (local address 127.5.122.3:35597)
0807 03:03:44.683987 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:44.684033 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:44.685808 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:44.686471 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:44.686478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:44.686557 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:44.687119 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:44.687136 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.688138 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.688142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:44.689331 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:44.689343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.690069 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.690077 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:44.690290 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:44.690906 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:03:44.690909 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:44.690944 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:03:44.691013 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:44.691367 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:44.691377 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:44.691384 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 03:03:44.691525 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 03:03:44.691881 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":305,"threads_started":1}
I20260807 03:03:44.694058 19776 heartbeater.cc:347] Connected to a master server at 127.5.122.62:37905
I20260807 03:03:44.694284 19776 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:44.695006 19776 heartbeater.cc:511] Master 127.5.122.62:37905 requested a full tablet report, sending...
I20260807 03:03:44.696195 19092 ts_manager.cc:194] Registered new tserver with Master: 0b7edb7a2d444fe8b80f308a9f2aef46 (127.5.122.3:41427)
I20260807 03:03:44.697041 19092 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.3:35597
I20260807 03:03:44.699271 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 03:03:44.705116 19393 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:44.703656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.5.122.62:37905)
0807 03:03:44.703870 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:44.703876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:44.704270 (+ 394us) server_negotiation.cc:407] Connection header received
0807 03:03:44.704407 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:44.704410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:44.704481 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:44.704551 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:44.704903 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57912: BlockingRecv error: recv got EOF from 127.0.0.1:57912 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:03:44.706012 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 19184
I20260807 03:03:44.713089 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:44.706843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47780 (local address 127.5.122.1:33705)
0807 03:03:44.707338 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:44.707346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:44.707375 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:03:44.707439 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:44.707445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:44.707636 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:44.707760 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:44.708058 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.709063 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.710226 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.710831 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.710925 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:44.711842 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:44.711848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:44.711860 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:44.712419 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:44.712464 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:44.712488 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:44.712574 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:44.712664 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 03:03:44.712791 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":260,"threads_started":1}
I20260807 03:03:44.714440 19382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:44.794010 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 19184
I20260807 03:03:44.812382 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 19396
I20260807 03:03:44.819852 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:44.813194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.5.122.2:40099)
0807 03:03:44.813654 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:44.813661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:44.813686 (+ 25us) server_negotiation.cc:407] Connection header received
0807 03:03:44.813743 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:44.813749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:44.813963 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:44.814097 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:44.814543 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.815518 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.816695 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.817252 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.817375 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:44.818476 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:44.818486 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:44.818497 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:44.819060 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:44.819106 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:44.819126 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:44.819234 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:44.819380 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 03:03:44.819510 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":219,"threads_started":1}
I20260807 03:03:44.820983 19574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:44.900949 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 19396
I20260807 03:03:44.922569 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 19584
I20260807 03:03:44.929912 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:44.923560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.5.122.3:41427)
0807 03:03:44.924128 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:44.924135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:44.924164 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:03:44.924223 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:44.924229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:44.924436 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:44.924560 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:44.924938 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.925930 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.927099 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:44.927638 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:44.927730 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:44.928623 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:44.928630 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:44.928642 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:44.929205 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:44.929250 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:44.929270 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:44.929361 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:44.929450 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 03:03:44.929558 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":276,"threads_started":1}
I20260807 03:03:44.931152 19766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:45.011554 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 19584
I20260807 03:03:45.034618 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 19043
I20260807 03:03:45.035893 19140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:45.137542 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 19043
2026-08-07T03:03:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2334 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T03:03:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:45Z Disabled control of system clock
I20260807 03:03:45.215485 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:38993
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:38993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 03:03:45.529376 19810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:45.529642 19810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:45.529734 19810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:45.536741 19810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:45.536840 19810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:45.536885 19810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:45.536939 19810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:45.536986 19810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:45.545985 19810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:38993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:38993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:45.547271 19810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:45.548990 19810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:45.560411 19817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:45.560356 19821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:45.560446 19818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:45.561761 19810 server_base.cc:1034] running on GCE node
I20260807 03:03:45.562510 19810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:45.563855 19810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:45.565074 19810 hybrid_clock.cc:648] HybridClock initialized: now 1786071825565027 us; error 48 us; skew 500 ppm
I20260807 03:03:45.565433 19810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:45.567556 19810 webserver.cc:460] Webserver started at http://127.5.122.62:38909/ using document root <none> and password file <none>
I20260807 03:03:45.568212 19810 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:45.568323 19810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:45.568637 19810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:45.571493 19810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "0371598930584f4aaa5b0c0525852c7b"
format_stamp: "Formatted at 2026-08-07 03:03:45 on dist-test-slave-fgb3"
server_key: "a79a6141d27782d4b771c13c9b9af6f9"
server_key_iv: "f4cd956028c630e5d5d570a4632dad43"
server_key_version: "encryptionkey@0"
I20260807 03:03:45.572247 19810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "0371598930584f4aaa5b0c0525852c7b"
format_stamp: "Formatted at 2026-08-07 03:03:45 on dist-test-slave-fgb3"
server_key: "a79a6141d27782d4b771c13c9b9af6f9"
server_key_iv: "f4cd956028c630e5d5d570a4632dad43"
server_key_version: "encryptionkey@0"
I20260807 03:03:45.577099 19810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 03:03:45.580368 19830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:45.581578 19810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:03:45.581727 19810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "0371598930584f4aaa5b0c0525852c7b"
format_stamp: "Formatted at 2026-08-07 03:03:45 on dist-test-slave-fgb3"
server_key: "a79a6141d27782d4b771c13c9b9af6f9"
server_key_iv: "f4cd956028c630e5d5d570a4632dad43"
server_key_version: "encryptionkey@0"
I20260807 03:03:45.581918 19810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:45.587364 19810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:45.588069 19810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:45.588346 19810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:45.604764 19810 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:38993
I20260807 03:03:45.604795 19910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:38993 every 8 connection(s)
I20260807 03:03:45.606076 19810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:45.609660 19912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:45.616146 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 19810
I20260807 03:03:45.616360 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:03:45.617024 5608 external_mini_cluster.cc:1428] Setting key 8db04b6bf85da8fe9d5beb16b1b0dcd3
I20260807 03:03:45.619779 19912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b: Bootstrap starting.
I20260807 03:03:45.623823 19912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:45.625200 19912 log.cc:824] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:45.626380 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:45.618968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.5.122.62:38993)
0807 03:03:45.620069 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:45.620084 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:45.620134 (+ 50us) server_negotiation.cc:407] Connection header received
0807 03:03:45.622579 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:45.622607 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:45.623133 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:45.623548 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:45.624569 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:45.624578 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:45.624592 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:45.624609 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:45.624677 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:45.624699 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:45.624897 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:45.625033 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 03:03:45.625302 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":353,"threads_started":1}
I20260807 03:03:45.628010 19912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b: No bootstrap required, opened a new log
I20260807 03:03:45.634902 19912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0371598930584f4aaa5b0c0525852c7b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38993 } }
I20260807 03:03:45.635232 19912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:45.635324 19912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0371598930584f4aaa5b0c0525852c7b, State: Initialized, Role: FOLLOWER
I20260807 03:03:45.635836 19912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0371598930584f4aaa5b0c0525852c7b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38993 } }
I20260807 03:03:45.636057 19912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:45.636183 19912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:45.636319 19912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:45.638465 19912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0371598930584f4aaa5b0c0525852c7b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38993 } }
I20260807 03:03:45.638937 19912 leader_election.cc:302] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0371598930584f4aaa5b0c0525852c7b; no voters:
I20260807 03:03:45.639514 19912 leader_election.cc:288] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:03:45.639663 19921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:45.640718 19921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [term 1 LEADER]: Becoming Leader. State: Replica: 0371598930584f4aaa5b0c0525852c7b, State: Running, Role: LEADER
I20260807 03:03:45.641237 19921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0371598930584f4aaa5b0c0525852c7b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38993 } }
I20260807 03:03:45.641613 19912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:45.644045 19923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0371598930584f4aaa5b0c0525852c7b. Latest consensus state: current_term: 1 leader_uuid: "0371598930584f4aaa5b0c0525852c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0371598930584f4aaa5b0c0525852c7b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38993 } } }
I20260807 03:03:45.644044 19922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0371598930584f4aaa5b0c0525852c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0371598930584f4aaa5b0c0525852c7b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38993 } } }
I20260807 03:03:45.644524 19923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:45.644526 19922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:45.647358 19932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:45.651932 19932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:45.660038 19932 catalog_manager.cc:1347] Generated new cluster ID: 94e896e96af644fcb981da50b2d5b29a
I20260807 03:03:45.660136 19932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:45.703541 19932 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:03:45.703986 19932 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:45.713066 19932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0371598930584f4aaa5b0c0525852c7b: Generated new TSK 0
I20260807 03:03:45.713984 19932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:03:45.727238 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:38993
--builtin_ntp_servers=127.5.122.20:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 03:03:46.049446 19947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:46.049710 19947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:46.049829 19947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:46.056962 19947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:46.057072 19947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:46.057212 19947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:46.066219 19947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:38993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:46.067484 19947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:46.069268 19947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:46.081661 19957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:46.081769 19954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:46.081703 19955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:46.082424 19947 server_base.cc:1034] running on GCE node
I20260807 03:03:46.083057 19947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:46.083961 19947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:46.085150 19947 hybrid_clock.cc:648] HybridClock initialized: now 1786071826085088 us; error 52 us; skew 500 ppm
I20260807 03:03:46.085453 19947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:46.087733 19947 webserver.cc:460] Webserver started at http://127.5.122.1:37481/ using document root <none> and password file <none>
I20260807 03:03:46.088392 19947 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:46.088495 19947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:46.088815 19947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:46.091581 19947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "3f7c9a586e5c421fa823668343bebec6"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "654d3f163afacc3370c895e4055c1b2f"
server_key_iv: "3c16bb526c334039e5072b8fbd83442b"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.092360 19947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "3f7c9a586e5c421fa823668343bebec6"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "654d3f163afacc3370c895e4055c1b2f"
server_key_iv: "3c16bb526c334039e5072b8fbd83442b"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.097131 19947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 03:03:46.100382 19970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.101567 19947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 03:03:46.101706 19947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "3f7c9a586e5c421fa823668343bebec6"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "654d3f163afacc3370c895e4055c1b2f"
server_key_iv: "3c16bb526c334039e5072b8fbd83442b"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.101907 19947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:46.107208 19947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:46.107913 19947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:46.108152 19947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:46.108987 19947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:46.110332 19947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:46.110437 19947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.110539 19947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:46.110611 19947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.140518 19947 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:33439
I20260807 03:03:46.140594 20142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:33439 every 8 connection(s)
I20260807 03:03:46.141830 19947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:46.143913 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 19947
I20260807 03:03:46.144114 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:03:46.144829 5608 external_mini_cluster.cc:1428] Setting key 4f67153c10d0e6195ae2bfce2f763105
I20260807 03:03:46.147768 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:38993
--builtin_ntp_servers=127.5.122.20:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 03:03:46.150949 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:46.144301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:57287 (local address 127.5.122.62:38993)
0807 03:03:46.144810 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:46.144816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:46.146645 (+ 1829us) server_negotiation.cc:407] Connection header received
0807 03:03:46.148826 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:46.148830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:46.148878 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:46.148966 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:46.150343 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:46.150350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:46.150355 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:46.150371 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:46.150411 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:46.150424 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:46.150518 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:46.150696 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 03:03:46.150742 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":252,"threads_started":1}
I20260807 03:03:46.152060 20147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:46.144918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:38993 (local address 127.5.122.1:57287)
0807 03:03:46.146490 (+ 1572us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:46.146544 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:46.148562 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:46.149208 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:46.149221 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:46.149347 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:46.150006 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:03:46.150009 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:46.150054 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:03:46.150155 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:46.150510 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:46.150520 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:46.150527 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 03:03:46.150782 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 03:03:46.151010 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"thread_start_us":816,"threads_started":1}
I20260807 03:03:46.153544 20144 heartbeater.cc:347] Connected to a master server at 127.5.122.62:38993
I20260807 03:03:46.153811 20144 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:46.154469 20144 heartbeater.cc:511] Master 127.5.122.62:38993 requested a full tablet report, sending...
I20260807 03:03:46.156472 19855 ts_manager.cc:194] Registered new tserver with Master: 3f7c9a586e5c421fa823668343bebec6 (127.5.122.1:33439)
W20260807 03:03:46.469213 20148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:46.469528 20148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:46.469667 20148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:46.476642 20148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:46.476753 20148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:46.476898 20148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:46.485548 20148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:38993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:46.486773 20148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:46.488394 20148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:46.500365 20157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:46.500403 20160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:46.500437 20158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:46.501343 20148 server_base.cc:1034] running on GCE node
I20260807 03:03:46.501885 20148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:46.502805 20148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:46.504007 20148 hybrid_clock.cc:648] HybridClock initialized: now 1786071826503956 us; error 38 us; skew 500 ppm
I20260807 03:03:46.504309 20148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:46.506428 20148 webserver.cc:460] Webserver started at http://127.5.122.2:38361/ using document root <none> and password file <none>
I20260807 03:03:46.507054 20148 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:46.507153 20148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:46.507455 20148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:46.512620 20148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "795c99d74c4f422a9e3c1f6c7c9b1632"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "ed70bd45e7828e7f63dcea9bc63bd1cd"
server_key_iv: "23d6dcd5d1165d465a7802765782289f"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.513500 20148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "795c99d74c4f422a9e3c1f6c7c9b1632"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "ed70bd45e7828e7f63dcea9bc63bd1cd"
server_key_iv: "23d6dcd5d1165d465a7802765782289f"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.518713 20148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 03:03:46.522058 20173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.523367 20148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:46.523518 20148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "795c99d74c4f422a9e3c1f6c7c9b1632"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "ed70bd45e7828e7f63dcea9bc63bd1cd"
server_key_iv: "23d6dcd5d1165d465a7802765782289f"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.523710 20148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:46.528959 20148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:46.529661 20148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:46.529901 20148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:46.530795 20148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:46.532099 20148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:46.532193 20148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.532294 20148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:46.532361 20148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.562914 20148 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:42599
I20260807 03:03:46.562994 20360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:42599 every 8 connection(s)
I20260807 03:03:46.564194 20148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:46.571695 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:46.566656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:51861 (local address 127.5.122.62:38993)
0807 03:03:46.566824 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:46.566828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:46.568153 (+ 1325us) server_negotiation.cc:407] Connection header received
0807 03:03:46.569724 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:46.569728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:46.569790 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:46.569873 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:46.571029 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:46.571033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:46.571036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:46.571054 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:46.571087 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:46.571099 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:46.571207 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:46.571383 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 03:03:46.571446 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:03:46.572429 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:46.567115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:38993 (local address 127.5.122.2:51861)
0807 03:03:46.567999 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:46.568039 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:46.569504 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:46.570045 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:46.570052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:46.570134 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:46.570743 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:03:46.570745 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:46.570781 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:03:46.570863 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:46.571174 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:46.571180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:46.571185 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:03:46.571389 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 03:03:46.571570 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":289,"threads_started":1}
I20260807 03:03:46.573736 20361 heartbeater.cc:347] Connected to a master server at 127.5.122.62:38993
I20260807 03:03:46.574002 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 20148
I20260807 03:03:46.574031 20361 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:46.574156 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:03:46.574622 5608 external_mini_cluster.cc:1428] Setting key c75a976fcda8a45549f6c0b1ec11fbe7
I20260807 03:03:46.574721 20361 heartbeater.cc:511] Master 127.5.122.62:38993 requested a full tablet report, sending...
I20260807 03:03:46.576223 19854 ts_manager.cc:194] Registered new tserver with Master: 795c99d74c4f422a9e3c1f6c7c9b1632 (127.5.122.2:42599)
I20260807 03:03:46.577371 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:38993
--builtin_ntp_servers=127.5.122.20:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 03:03:46.897161 20366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:46.897408 20366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:46.897531 20366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:46.904465 20366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:46.904551 20366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:46.904696 20366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:46.913349 20366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:38993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:46.914552 20366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:46.916178 20366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:46.928218 20380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:46.928721 20379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:46.928256 20383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:46.931573 20366 server_base.cc:1034] running on GCE node
I20260807 03:03:46.932116 20366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:46.932979 20366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:46.934162 20366 hybrid_clock.cc:648] HybridClock initialized: now 1786071826934122 us; error 36 us; skew 500 ppm
I20260807 03:03:46.934469 20366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:03:46.936496 20366 webserver.cc:460] Webserver started at http://127.5.122.3:35019/ using document root <none> and password file <none>
I20260807 03:03:46.937105 20366 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:46.937196 20366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:46.937498 20366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:46.940217 20366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "e3b8068c055c481ba535a06d7d193519"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "dea53abaf651a3ce7b6aa553c0a3c504"
server_key_iv: "693d8a92fedbf444bdeb22596b960514"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.940923 20366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "e3b8068c055c481ba535a06d7d193519"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "dea53abaf651a3ce7b6aa553c0a3c504"
server_key_iv: "693d8a92fedbf444bdeb22596b960514"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.945679 20366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260807 03:03:46.949030 20393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.950274 20366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:03:46.950451 20366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "e3b8068c055c481ba535a06d7d193519"
format_stamp: "Formatted at 2026-08-07 03:03:46 on dist-test-slave-fgb3"
server_key: "dea53abaf651a3ce7b6aa553c0a3c504"
server_key_iv: "693d8a92fedbf444bdeb22596b960514"
server_key_version: "encryptionkey@0"
I20260807 03:03:46.950647 20366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:46.955848 20366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:46.956604 20366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:46.956843 20366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:46.957746 20366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:46.959102 20366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:46.959196 20366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.959308 20366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:46.959378 20366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:46.988878 20366 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36023
I20260807 03:03:46.988956 20562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36023 every 8 connection(s)
I20260807 03:03:46.990126 20366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:46.992558 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 20366
I20260807 03:03:46.992749 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:03:46.993196 5608 external_mini_cluster.cc:1428] Setting key f48f1090dc7b89e451408f79ea89ef2e
I20260807 03:03:46.998505 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:46.992613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:51671 (local address 127.5.122.62:38993)
0807 03:03:46.992826 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:46.992831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:46.994458 (+ 1627us) server_negotiation.cc:407] Connection header received
0807 03:03:46.996179 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:46.996185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:46.996255 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:46.996350 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:46.997666 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:46.997678 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:46.997684 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:46.997705 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:46.997748 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:46.997762 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:46.997871 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:46.998088 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 03:03:46.998159 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:03:46.999321 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:46.993139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:38993 (local address 127.5.122.3:51671)
0807 03:03:46.994223 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:46.994306 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:46.995917 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:46.996590 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:46.996598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:46.996682 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:46.997366 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:03:46.997369 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:46.997407 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:03:46.997492 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:46.997857 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:46.997869 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:46.997878 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 03:03:46.998160 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 03:03:46.998433 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":343,"threads_started":1}
I20260807 03:03:47.000519 20563 heartbeater.cc:347] Connected to a master server at 127.5.122.62:38993
I20260807 03:03:47.000759 20563 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:47.001262 20563 heartbeater.cc:511] Master 127.5.122.62:38993 requested a full tablet report, sending...
I20260807 03:03:47.002503 19854 ts_manager.cc:194] Registered new tserver with Master: e3b8068c055c481ba535a06d7d193519 (127.5.122.3:36023)
I20260807 03:03:47.007861 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 03:03:47.013533 20146 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:03:47.012226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.5.122.62:38993)
0807 03:03:47.012393 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:47.012397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:47.012828 (+ 431us) server_negotiation.cc:407] Connection header received
0807 03:03:47.012965 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:47.012968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:47.013017 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:47.013094 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:47.013348 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39458: BlockingRecv error: recv got EOF from 127.0.0.1:39458 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:03:47.014169 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 19947
I20260807 03:03:47.024140 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:47.015479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.5.122.1:33439)
0807 03:03:47.015926 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:47.015933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:47.021978 (+ 6045us) server_negotiation.cc:407] Connection header received
0807 03:03:47.022076 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:47.022083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:47.022300 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:47.022454 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:47.023423 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:47.023433 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:47.023448 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:47.023466 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:47.023520 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:47.023539 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:47.023644 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:47.023757 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 03:03:47.023846 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":194,"threads_started":1}
I20260807 03:03:47.025368 20134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:47.105042 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 19947
I20260807 03:03:47.124682 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 20148
I20260807 03:03:47.128216 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:47.125605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.5.122.2:42599)
0807 03:03:47.126179 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:47.126186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:47.126213 (+ 27us) server_negotiation.cc:407] Connection header received
0807 03:03:47.126277 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:47.126284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:47.126554 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:47.126678 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:47.127600 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:47.127606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:47.127617 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:47.127635 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:47.127677 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:47.127691 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:47.127790 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:47.127868 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 03:03:47.127950 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":293,"threads_started":1}
I20260807 03:03:47.129441 20354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:47.208981 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 20148
I20260807 03:03:47.228344 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 20366
I20260807 03:03:47.232066 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:47.229352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51778 (local address 127.5.122.3:36023)
0807 03:03:47.230061 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:47.230068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:47.230099 (+ 31us) server_negotiation.cc:407] Connection header received
0807 03:03:47.230159 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:47.230165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:47.230418 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:47.230539 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:47.231405 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:47.231411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:47.231422 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:03:47.231438 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:47.231478 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:03:47.231491 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:47.231616 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:47.231666 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 03:03:47.231753 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":390,"threads_started":1}
I20260807 03:03:47.233201 20557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:47.319839 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 20366
I20260807 03:03:47.338762 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 19810
I20260807 03:03:47.339962 19902 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:47.439752 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 19810
2026-08-07T03:03:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2301 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:03:47 dist-test-slave-fgb3 krb5kdc[20593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:03:47 dist-test-slave-fgb3 krb5kdc[20593](info): set up 2 sockets
Aug 07 03:03:47 dist-test-slave-fgb3 krb5kdc[20593](info): commencing operation
krb5kdc: starting...
W20260807 03:03:48.615200 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:03:48 dist-test-slave-fgb3 krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:03:48.845270 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40013
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:49.158020 20615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:49.158344 20615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:49.158497 20615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:49.165544 20615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:03:49.165650 20615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:49.165730 20615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:49.165807 20615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:03:49.165869 20615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:03:49.175489 20615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:40013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:49.176702 20615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:49.178004 20615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:49.188943 20626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:49.188910 20627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:49.188966 20631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:49.189682 20615 server_base.cc:1034] running on GCE node
I20260807 03:03:49.190479 20615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:49.191769 20615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:49.192965 20615 hybrid_clock.cc:648] HybridClock initialized: now 1786071829192918 us; error 33 us; skew 500 ppm
I20260807 03:03:49.193269 20615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:49 dist-test-slave-fgb3 krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071829, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:49.199795 20615 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:03:49.201156 20615 webserver.cc:460] Webserver started at http://127.5.122.62:43819/ using document root <none> and password file <none>
I20260807 03:03:49.201771 20615 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:49.201894 20615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:49.202189 20615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:49.205061 20615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "f1ee576522914b2888f6ea87a3fa14fa"
format_stamp: "Formatted at 2026-08-07 03:03:49 on dist-test-slave-fgb3"
server_key: "9fad7d80b919aba553a4b28071b053c6"
server_key_iv: "55b3f15a75774363786cd3aface7515b"
server_key_version: "encryptionkey@0"
I20260807 03:03:49.205789 20615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "f1ee576522914b2888f6ea87a3fa14fa"
format_stamp: "Formatted at 2026-08-07 03:03:49 on dist-test-slave-fgb3"
server_key: "9fad7d80b919aba553a4b28071b053c6"
server_key_iv: "55b3f15a75774363786cd3aface7515b"
server_key_version: "encryptionkey@0"
I20260807 03:03:49.210726 20615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 03:03:49.213899 20643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:49.215158 20615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:03:49.215305 20615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "f1ee576522914b2888f6ea87a3fa14fa"
format_stamp: "Formatted at 2026-08-07 03:03:49 on dist-test-slave-fgb3"
server_key: "9fad7d80b919aba553a4b28071b053c6"
server_key_iv: "55b3f15a75774363786cd3aface7515b"
server_key_version: "encryptionkey@0"
I20260807 03:03:49.215476 20615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:49.237358 20615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:49.241032 20615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:49.241281 20615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:49.258165 20615 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:40013
I20260807 03:03:49.258185 20719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:40013 every 8 connection(s)
I20260807 03:03:49.259780 20615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:03:49.263291 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 20615
I20260807 03:03:49.263520 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:03:49.263625 20722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:49.264329 5608 external_mini_cluster.cc:1428] Setting key b58757aa9333818f798e98aa5b9a79ec
I20260807 03:03:49.278036 20722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa: Bootstrap starting.
Aug 07 03:03:49 dist-test-slave-fgb3 krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:49.281262 20722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:49.282451 20722 log.cc:824] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:49.285169 20722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa: No bootstrap required, opened a new log
I20260807 03:03:49.287029 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:49.266386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.5.122.62:40013)
0807 03:03:49.267142 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:49.267152 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:49.267190 (+ 38us) server_negotiation.cc:407] Connection header received
0807 03:03:49.268634 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:49.268654 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:49.269024 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:49.269322 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:49.271105 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:49.272333 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:49.274106 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:49.275203 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:49.275405 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:49.280213 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:49.280221 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:49.280233 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:49.280265 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:49.283283 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:49.283908 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:49.283915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:49.283921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:49.284083 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:49.284455 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:49.284458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:49.284460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:49.284950 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:49.285240 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:49.285655 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 03:03:49.286039 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":340,"threads_started":1}
I20260807 03:03:49.292621 20722 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ee576522914b2888f6ea87a3fa14fa" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40013 } }
I20260807 03:03:49.292975 20722 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:49.293066 20722 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ee576522914b2888f6ea87a3fa14fa, State: Initialized, Role: FOLLOWER
I20260807 03:03:49.293574 20722 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ee576522914b2888f6ea87a3fa14fa" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40013 } }
I20260807 03:03:49.293784 20722 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:03:49.293905 20722 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:03:49.294052 20722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:49.296298 20722 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ee576522914b2888f6ea87a3fa14fa" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40013 } }
I20260807 03:03:49.296770 20722 leader_election.cc:302] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1ee576522914b2888f6ea87a3fa14fa; no voters:
I20260807 03:03:49.297341 20722 leader_election.cc:288] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:03:49.297528 20729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:49.298437 20729 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [term 1 LEADER]: Becoming Leader. State: Replica: f1ee576522914b2888f6ea87a3fa14fa, State: Running, Role: LEADER
I20260807 03:03:49.298924 20729 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ee576522914b2888f6ea87a3fa14fa" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40013 } }
I20260807 03:03:49.299441 20722 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:03:49.301621 20733 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1ee576522914b2888f6ea87a3fa14fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ee576522914b2888f6ea87a3fa14fa" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40013 } } }
I20260807 03:03:49.301620 20734 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1ee576522914b2888f6ea87a3fa14fa. Latest consensus state: current_term: 1 leader_uuid: "f1ee576522914b2888f6ea87a3fa14fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ee576522914b2888f6ea87a3fa14fa" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40013 } } }
I20260807 03:03:49.302008 20734 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:49.302008 20733 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa [sys.catalog]: This master's current role is: LEADER
I20260807 03:03:49.303352 20746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:03:49.308117 20746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:03:49.321710 20746 catalog_manager.cc:1347] Generated new cluster ID: bbcacd852b9e46fe9b202fb4c19b46a2
I20260807 03:03:49.321810 20746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:03:49.333720 20746 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:03:49.334964 20746 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:03:49.352061 20746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa: Generated new TSK 0
I20260807 03:03:49.352828 20746 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:03:49.423744 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40013
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:49.741076 20768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:49.741336 20768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:49.741437 20768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:49.748383 20768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:49.748479 20768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:49.748595 20768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:03:49.757741 20768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:49.758879 20768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:49.760040 20768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:49.771950 20784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:49.772017 20780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:49.772444 20778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:49.772691 20768 server_base.cc:1034] running on GCE node
I20260807 03:03:49.773242 20768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:49.774103 20768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:49.775311 20768 hybrid_clock.cc:648] HybridClock initialized: now 1786071829775233 us; error 69 us; skew 500 ppm
I20260807 03:03:49.775602 20768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:49 dist-test-slave-fgb3 krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071829, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:49.778736 20768 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:03:49.780439 20768 webserver.cc:460] Webserver started at http://127.5.122.1:37935/ using document root <none> and password file <none>
I20260807 03:03:49.781085 20768 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:49.781203 20768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:49.781494 20768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:49.784400 20768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "04426a113741464cbd3c7a3eed158a01"
format_stamp: "Formatted at 2026-08-07 03:03:49 on dist-test-slave-fgb3"
server_key: "54299a46f22d8306583ac72d9d131533"
server_key_iv: "66e6b3ea61303513edc7e234291c509d"
server_key_version: "encryptionkey@0"
I20260807 03:03:49.785233 20768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "04426a113741464cbd3c7a3eed158a01"
format_stamp: "Formatted at 2026-08-07 03:03:49 on dist-test-slave-fgb3"
server_key: "54299a46f22d8306583ac72d9d131533"
server_key_iv: "66e6b3ea61303513edc7e234291c509d"
server_key_version: "encryptionkey@0"
I20260807 03:03:49.790076 20768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 03:03:49.793313 20792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:49.794600 20768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:03:49.794745 20768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "04426a113741464cbd3c7a3eed158a01"
format_stamp: "Formatted at 2026-08-07 03:03:49 on dist-test-slave-fgb3"
server_key: "54299a46f22d8306583ac72d9d131533"
server_key_iv: "66e6b3ea61303513edc7e234291c509d"
server_key_version: "encryptionkey@0"
I20260807 03:03:49.794950 20768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:49.820060 20768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:49.824039 20768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:49.824304 20768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:49.825250 20768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:49.826613 20768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:49.826694 20768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:49.826818 20768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:49.826874 20768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:49.856964 20768 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:36835
I20260807 03:03:49.857024 20967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:36835 every 8 connection(s)
I20260807 03:03:49.858247 20768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:03:49.867910 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 20768
I20260807 03:03:49.868147 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:03:49.868860 5608 external_mini_cluster.cc:1428] Setting key 7e03b06cd807a92c7210ed07b7393f19
Aug 07 03:03:49 dist-test-slave-fgb3 krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071829, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:49.877164 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:49.860666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:60819 (local address 127.5.122.62:40013)
0807 03:03:49.861140 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:49.861147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:49.862014 (+ 867us) server_negotiation.cc:407] Connection header received
0807 03:03:49.863831 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:49.863835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:49.863911 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:49.863999 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:49.865365 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:49.866069 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:49.867576 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:49.868177 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:49.868317 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:49.871978 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:49.871983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:49.871986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:49.872017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:49.874393 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:49.875154 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:49.875158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:49.875160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:49.875294 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:49.875697 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:49.875700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:49.875702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:49.875937 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:49.876052 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:49.876759 (+ 707us) server_negotiation.cc:299] Negotiation successful
0807 03:03:49.876916 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":216,"threads_started":1}
I20260807 03:03:49.878424 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:49.861149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40013 (local address 127.5.122.1:60819)
0807 03:03:49.861856 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:49.861899 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:49.863580 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:49.864178 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:49.864185 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:49.864689 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:49.865228 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:49.865245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:49.866171 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:49.866174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:49.867430 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:49.867442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:49.868413 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:49.868418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:49.868605 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:49.869251 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:49.869280 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:49.871774 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:49.874529 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:49.874537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:49.874550 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:49.875059 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:49.875386 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:49.875390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:49.875393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:49.875610 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:49.876051 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:49.876058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:49.876414 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 03:03:49.876678 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 03:03:49.877157 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":273,"threads_started":1}
I20260807 03:03:49.879834 20969 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40013
I20260807 03:03:49.880162 20969 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:49.880968 20969 heartbeater.cc:511] Master 127.5.122.62:40013 requested a full tablet report, sending...
I20260807 03:03:49.883082 20670 ts_manager.cc:194] Registered new tserver with Master: 04426a113741464cbd3c7a3eed158a01 (127.5.122.1:36835)
I20260807 03:03:49.885006 20670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:60819
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:03:49.924693 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40013
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:50.232873 20977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:50.233121 20977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:50.233242 20977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:50.240201 20977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:50.240324 20977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:50.240477 20977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:03:50.249607 20977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:50.250813 20977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:50.252018 20977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:50.264175 20986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:50.264724 20990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:50.264720 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:50.265172 20977 server_base.cc:1034] running on GCE node
I20260807 03:03:50.265781 20977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:50.266685 20977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:50.267882 20977 hybrid_clock.cc:648] HybridClock initialized: now 1786071830267833 us; error 41 us; skew 500 ppm
I20260807 03:03:50.268167 20977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:50 dist-test-slave-fgb3 krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071830, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:50.271333 20977 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:03:50.272895 20977 webserver.cc:460] Webserver started at http://127.5.122.2:38887/ using document root <none> and password file <none>
I20260807 03:03:50.273650 20977 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:50.273774 20977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:50.274091 20977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:50.277297 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "7badc2b502bc46d29a0a02f4f274bb7e"
format_stamp: "Formatted at 2026-08-07 03:03:50 on dist-test-slave-fgb3"
server_key: "e87ce9a6aa5074604bb5ceb7393c801f"
server_key_iv: "a46314556c9920ee42db95ddb8eeafa0"
server_key_version: "encryptionkey@0"
I20260807 03:03:50.278268 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "7badc2b502bc46d29a0a02f4f274bb7e"
format_stamp: "Formatted at 2026-08-07 03:03:50 on dist-test-slave-fgb3"
server_key: "e87ce9a6aa5074604bb5ceb7393c801f"
server_key_iv: "a46314556c9920ee42db95ddb8eeafa0"
server_key_version: "encryptionkey@0"
I20260807 03:03:50.283623 20977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:03:50.287022 21007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:50.288300 20977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 03:03:50.288455 20977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "7badc2b502bc46d29a0a02f4f274bb7e"
format_stamp: "Formatted at 2026-08-07 03:03:50 on dist-test-slave-fgb3"
server_key: "e87ce9a6aa5074604bb5ceb7393c801f"
server_key_iv: "a46314556c9920ee42db95ddb8eeafa0"
server_key_version: "encryptionkey@0"
I20260807 03:03:50.288635 20977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:50.320384 20977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:50.324280 20977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:50.324568 20977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:50.325546 20977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:50.326896 20977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:50.326972 20977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:50.327092 20977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:50.327143 20977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:50.356977 20977 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:38449
I20260807 03:03:50.357017 21182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:38449 every 8 connection(s)
I20260807 03:03:50.358237 20977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:03:50.364015 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 20977
I20260807 03:03:50.364207 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:03:50.364615 5608 external_mini_cluster.cc:1428] Setting key c256c38c807a5e4a619fe49d1316aa35
Aug 07 03:03:50 dist-test-slave-fgb3 krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071830, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:50.377575 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.360683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:57655 (local address 127.5.122.62:40013)
0807 03:03:50.360854 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:50.360858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:50.361957 (+ 1099us) server_negotiation.cc:407] Connection header received
0807 03:03:50.363803 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:50.363810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:50.363891 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:50.363983 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:50.365622 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.366472 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.368117 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.368707 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.368835 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.372396 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:50.372401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:50.372404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:50.372436 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:50.374825 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:50.375579 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:50.375582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:50.375584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:50.375698 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:50.376116 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:50.376119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:50.376121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:50.376353 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:50.376452 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:50.377181 (+ 729us) server_negotiation.cc:299] Negotiation successful
0807 03:03:50.377343 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:03:50.378715 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.361092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40013 (local address 127.5.122.2:57655)
0807 03:03:50.361779 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:50.361830 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:50.363535 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:50.364274 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:50.364287 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:50.364825 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:50.365469 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.365490 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.366615 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.366619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.367972 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.367985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.368940 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.368946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.369139 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.369762 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:50.369791 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:50.372173 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:50.374952 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:50.374961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:50.374976 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:50.375483 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:50.375785 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:50.375789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:50.375792 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:50.376026 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:50.376459 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:50.376467 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:50.376834 (+ 367us) client_negotiation.cc:769] Sending connection context
0807 03:03:50.377130 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 03:03:50.377599 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":280,"threads_started":1}
I20260807 03:03:50.380066 21183 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40013
I20260807 03:03:50.380347 21183 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:50.381040 21183 heartbeater.cc:511] Master 127.5.122.62:40013 requested a full tablet report, sending...
I20260807 03:03:50.382437 20670 ts_manager.cc:194] Registered new tserver with Master: 7badc2b502bc46d29a0a02f4f274bb7e (127.5.122.2:38449)
I20260807 03:03:50.383212 20670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:57655
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:03:50.421509 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40013
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:03:50.733374 21190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:03:50.733633 21190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:03:50.733736 21190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:03:50.740846 21190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:03:50.740957 21190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:03:50.741075 21190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:03:50.750494 21190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:03:50.751658 21190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:03:50.752918 21190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:03:50.766640 21197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:50.766685 21198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:03:50.766623 21201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:03:50.767354 21190 server_base.cc:1034] running on GCE node
I20260807 03:03:50.767928 21190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:03:50.768812 21190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:03:50.770012 21190 hybrid_clock.cc:648] HybridClock initialized: now 1786071830769953 us; error 51 us; skew 500 ppm
I20260807 03:03:50.770299 21190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:03:50 dist-test-slave-fgb3 krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071830, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:03:50.780398 21190 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:03:50.781764 21190 webserver.cc:460] Webserver started at http://127.5.122.3:34151/ using document root <none> and password file <none>
I20260807 03:03:50.782469 21190 fs_manager.cc:359] Metadata directory not provided
I20260807 03:03:50.782591 21190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:03:50.782881 21190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:03:50.785722 21190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "c63b873446b045559ca45af41583d2fb"
format_stamp: "Formatted at 2026-08-07 03:03:50 on dist-test-slave-fgb3"
server_key: "d645a98773d52917b13ea3f7668e1bda"
server_key_iv: "33caaaaa7a5518c84e0314399209e5bd"
server_key_version: "encryptionkey@0"
I20260807 03:03:50.786629 21190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "c63b873446b045559ca45af41583d2fb"
format_stamp: "Formatted at 2026-08-07 03:03:50 on dist-test-slave-fgb3"
server_key: "d645a98773d52917b13ea3f7668e1bda"
server_key_iv: "33caaaaa7a5518c84e0314399209e5bd"
server_key_version: "encryptionkey@0"
I20260807 03:03:50.791507 21190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 03:03:50.794736 21210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:50.795985 21190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 03:03:50.796128 21190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "c63b873446b045559ca45af41583d2fb"
format_stamp: "Formatted at 2026-08-07 03:03:50 on dist-test-slave-fgb3"
server_key: "d645a98773d52917b13ea3f7668e1bda"
server_key_iv: "33caaaaa7a5518c84e0314399209e5bd"
server_key_version: "encryptionkey@0"
I20260807 03:03:50.796296 21190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:03:50.833202 21190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:03:50.837546 21190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:03:50.837875 21190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:03:50.838984 21190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:03:50.840458 21190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:03:50.840555 21190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:50.840690 21190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:03:50.840754 21190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:03:50.887910 20969 heartbeater.cc:503] Master 127.5.122.62:40013 was elected leader, sending a full tablet report...
I20260807 03:03:50.888096 21190 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:43081
I20260807 03:03:50.888152 21395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:43081 every 8 connection(s)
I20260807 03:03:50.889783 21190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:03:50.894805 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 21190
I20260807 03:03:50.895009 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:03:50.895524 5608 external_mini_cluster.cc:1428] Setting key fc6f83ad59ff033d9b1489dd4ca431f0
Aug 07 03:03:50 dist-test-slave-fgb3 krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071830, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:03:50.913795 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.892926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:55673 (local address 127.5.122.62:40013)
0807 03:03:50.893388 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:50.893392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:50.894512 (+ 1120us) server_negotiation.cc:407] Connection header received
0807 03:03:50.897073 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:50.897078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:50.897160 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:50.897306 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:50.899601 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.900905 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.903266 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.903923 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.904052 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.907800 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:50.907806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:50.907809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:50.907846 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:50.910987 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:50.911772 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:50.911775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:50.911778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:50.911904 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:50.912298 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:50.912301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:50.912302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:50.912556 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:50.912663 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:50.913315 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 03:03:50.913482 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":226,"threads_started":1}
I20260807 03:03:50.914944 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.893484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40013 (local address 127.5.122.3:55673)
0807 03:03:50.894340 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:50.894399 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:50.896666 (+ 2267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:50.897567 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:50.897575 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:50.898435 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:03:50.899387 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.899435 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.901056 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.901062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.903101 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.903119 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.904141 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.904149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.904459 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.905203 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:03:50.905228 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:03:50.907582 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:03:50.911144 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:50.911154 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:50.911170 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:50.911652 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:50.911988 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:03:50.911991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:03:50.911993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:03:50.912186 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:03:50.912658 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:03:50.912664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:03:50.912991 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 03:03:50.913260 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 03:03:50.913698 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":297,"threads_started":1}
I20260807 03:03:50.916303 21396 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40013
I20260807 03:03:50.916615 21396 heartbeater.cc:464] Registering TS with master...
I20260807 03:03:50.917380 21396 heartbeater.cc:511] Master 127.5.122.62:40013 requested a full tablet report, sending...
I20260807 03:03:50.918869 20670 ts_manager.cc:194] Registered new tserver with Master: c63b873446b045559ca45af41583d2fb (127.5.122.3:43081)
I20260807 03:03:50.919713 20670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:55673
I20260807 03:03:50.923581 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:03:50.939982 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.928067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.5.122.62:40013)
0807 03:03:50.928297 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:50.928304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:50.928747 (+ 443us) server_negotiation.cc:407] Connection header received
0807 03:03:50.928880 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:50.928884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:50.928950 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:50.929033 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:50.930458 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.931203 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.932370 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.932816 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.932915 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.935017 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:50.935023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:50.935026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:50.935058 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:50.937512 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:50.938109 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:50.938113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:50.938115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:50.938262 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:50.938675 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:50.938681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:50.938684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:50.939019 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:50.939205 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:50.939513 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 03:03:50.939676 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:03:50.943897 20670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:03:50.950642 20670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:03:50.972638 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.962057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60022 (local address 127.5.122.2:38449)
0807 03:03:50.962741 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:50.962750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:50.963396 (+ 646us) server_negotiation.cc:407] Connection header received
0807 03:03:50.963824 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:50.963833 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:50.964067 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:50.964243 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:50.965651 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.967331 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.969999 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.971452 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.971580 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.971753 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:50.972126 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 03:03:50.972283 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":340,"threads_started":1}
I20260807 03:03:50.972671 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.961750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:38449 (local address 127.0.0.1:60022)
0807 03:03:50.963229 (+ 1479us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:50.963286 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:50.963549 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:50.964563 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:50.964572 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:50.964605 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:50.965189 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.965202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.967492 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.967502 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.969820 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.969832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.971659 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.971666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.971887 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.971939 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 03:03:50.972064 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 03:03:50.972249 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1237,"mutex_wait_us":93,"thread_start_us":1425,"threads_started":1}
I20260807 03:03:50.972743 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.962390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.5.122.3:43081)
0807 03:03:50.963222 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:50.963229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:50.963764 (+ 535us) server_negotiation.cc:407] Connection header received
0807 03:03:50.963878 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:50.963885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:50.964122 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:50.964311 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:50.965642 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.967223 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.970373 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.971371 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.971448 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.971650 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:50.972241 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 03:03:50.972413 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":517,"threads_started":1}
I20260807 03:03:50.973352 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.962024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:43081 (local address 127.0.0.1:51296)
0807 03:03:50.963495 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:50.963512 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:50.963636 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:50.964621 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:50.964626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:50.964654 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:50.965459 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.965472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.967448 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.967453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.970162 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.970177 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.971840 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.971846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.972047 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.972079 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:03:50.972950 (+ 871us) client_negotiation.cc:240] Negotiation successful
0807 03:03:50.973044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":140,"thread_start_us":1749,"threads_started":1}
I20260807 03:03:50.975033 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.964087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:36835 (local address 127.0.0.1:59122)
0807 03:03:50.965514 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:50.965528 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:50.965633 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:50.968072 (+ 2439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:50.968077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:50.968103 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:50.968652 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.968665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.970554 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.970559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.972497 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:50.972512 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.974490 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.974495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:50.974606 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.974628 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 03:03:50.974704 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 03:03:50.974786 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1312,"mutex_wait_us":33,"thread_start_us":1181,"threads_started":1}
I20260807 03:03:50.975212 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:50.964354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59122 (local address 127.5.122.1:36835)
0807 03:03:50.967502 (+ 3148us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:50.967510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:50.967536 (+ 26us) server_negotiation.cc:407] Connection header received
0807 03:03:50.967633 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:50.967640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:50.967870 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:50.968158 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:50.968894 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.970415 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.972688 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:50.974280 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:50.974436 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:50.974648 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:50.974752 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 03:03:50.974912 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2977,"thread_start_us":2810,"threads_started":1}
I20260807 03:03:50.976794 21079 tablet_service.cc:1467] Processing CreateTablet for tablet a2c3b901d4e14cf3b94b50878ea6e117 (DEFAULT_TABLE table=test-table [id=e7b45545e4ee4bd484a11209019c2180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:50.978085 21079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c3b901d4e14cf3b94b50878ea6e117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:50.980738 21292 tablet_service.cc:1467] Processing CreateTablet for tablet a2c3b901d4e14cf3b94b50878ea6e117 (DEFAULT_TABLE table=test-table [id=e7b45545e4ee4bd484a11209019c2180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:50.980737 20863 tablet_service.cc:1467] Processing CreateTablet for tablet a2c3b901d4e14cf3b94b50878ea6e117 (DEFAULT_TABLE table=test-table [id=e7b45545e4ee4bd484a11209019c2180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:03:50.982129 20863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c3b901d4e14cf3b94b50878ea6e117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:50.982138 21292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c3b901d4e14cf3b94b50878ea6e117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:03:50.989264 21425 tablet_bootstrap.cc:492] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Bootstrap starting.
I20260807 03:03:50.992761 21425 tablet_bootstrap.cc:654] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:50.993772 21425 log.cc:824] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:50.996381 21425 tablet_bootstrap.cc:492] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: No bootstrap required, opened a new log
I20260807 03:03:50.996459 21427 tablet_bootstrap.cc:492] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Bootstrap starting.
I20260807 03:03:50.996737 21425 ts_tablet_manager.cc:1397] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260807 03:03:50.997167 21429 tablet_bootstrap.cc:492] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Bootstrap starting.
I20260807 03:03:51.001016 21429 tablet_bootstrap.cc:654] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:51.001897 21427 tablet_bootstrap.cc:654] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Neither blocks nor log segments found. Creating new log.
I20260807 03:03:51.002056 21429 log.cc:824] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:51.003054 21427 log.cc:824] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Log is configured to *not* fsync() on all Append() calls
I20260807 03:03:51.004792 21429 tablet_bootstrap.cc:492] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: No bootstrap required, opened a new log
I20260807 03:03:51.005035 21429 ts_tablet_manager.cc:1397] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 03:03:51.005985 21427 tablet_bootstrap.cc:492] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: No bootstrap required, opened a new log
I20260807 03:03:51.006227 21427 ts_tablet_manager.cc:1397] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260807 03:03:51.006793 21425 raft_consensus.cc:348] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.007225 21425 raft_consensus.cc:374] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:51.007337 21425 raft_consensus.cc:729] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7badc2b502bc46d29a0a02f4f274bb7e, State: Initialized, Role: FOLLOWER
I20260807 03:03:51.007956 21425 consensus_queue.cc:260] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.009039 21183 heartbeater.cc:503] Master 127.5.122.62:40013 was elected leader, sending a full tablet report...
I20260807 03:03:51.009583 21425 ts_tablet_manager.cc:1428] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 03:03:51.013728 21429 raft_consensus.cc:348] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.014113 21429 raft_consensus.cc:374] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:51.014266 21429 raft_consensus.cc:729] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04426a113741464cbd3c7a3eed158a01, State: Initialized, Role: FOLLOWER
I20260807 03:03:51.014900 21429 consensus_queue.cc:260] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.015106 21427 raft_consensus.cc:348] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.015494 21427 raft_consensus.cc:374] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:03:51.015592 21427 raft_consensus.cc:729] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c63b873446b045559ca45af41583d2fb, State: Initialized, Role: FOLLOWER
I20260807 03:03:51.016142 21427 consensus_queue.cc:260] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.016528 21429 ts_tablet_manager.cc:1428] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 03:03:51.017158 21396 heartbeater.cc:503] Master 127.5.122.62:40013 was elected leader, sending a full tablet report...
I20260807 03:03:51.017678 21427 ts_tablet_manager.cc:1428] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260807 03:03:51.110142 21185 tablet.cc:2369] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:03:51.110921 20970 tablet.cc:2369] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:03:51.141746 21397 tablet.cc:2369] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:03:51.164539 21435 raft_consensus.cc:482] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:03:51.164853 21435 raft_consensus.cc:504] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.166608 21435 leader_election.cc:288] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7badc2b502bc46d29a0a02f4f274bb7e (127.5.122.2:38449), c63b873446b045559ca45af41583d2fb (127.5.122.3:43081)
I20260807 03:03:51.173599 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.167128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:35787 (local address 127.5.122.3:43081)
0807 03:03:51.167334 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:51.167341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:51.168122 (+ 781us) server_negotiation.cc:407] Connection header received
0807 03:03:51.168395 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:51.168402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:51.168513 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:51.168604 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:51.169394 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.170141 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.171906 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.172651 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.172745 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.172844 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:51.173231 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 03:03:51.173360 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:03:51.173641 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.167054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:47175 (local address 127.5.122.2:38449)
0807 03:03:51.167283 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:51.167290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:51.167952 (+ 662us) server_negotiation.cc:407] Connection header received
0807 03:03:51.168186 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:51.168193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:51.168273 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:51.168335 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:03:51.169177 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.169941 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.171912 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.172716 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.172855 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.172948 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:51.173260 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 03:03:51.173393 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 03:03:51.173763 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.166779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:38449 (local address 127.5.122.1:47175)
0807 03:03:51.167840 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:51.167856 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:51.168007 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:51.168543 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:51.168548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:51.168573 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:51.169009 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:51.169020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.170069 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.170074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:51.171750 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:51.171762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.172946 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.172950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:51.173111 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.173134 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:03:51.173217 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 03:03:51.173351 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"mutex_wait_us":73,"thread_start_us":707,"threads_started":1}
I20260807 03:03:51.174082 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.167077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:43081 (local address 127.5.122.1:35787)
0807 03:03:51.167982 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0807 03:03:51.168028 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 03:03:51.168199 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:03:51.168774 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:03:51.168778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:03:51.168798 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:03:51.169233 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:51.169242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.170278 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.170282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:51.171753 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:03:51.171766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.172863 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.172878 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:03:51.173051 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.173108 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 03:03:51.173178 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 03:03:51.173297 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":6,"thread_start_us":476,"threads_started":1}
I20260807 03:03:51.174813 21113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3b901d4e14cf3b94b50878ea6e117" candidate_uuid: "04426a113741464cbd3c7a3eed158a01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" is_pre_election: true
I20260807 03:03:51.174968 21326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3b901d4e14cf3b94b50878ea6e117" candidate_uuid: "04426a113741464cbd3c7a3eed158a01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c63b873446b045559ca45af41583d2fb" is_pre_election: true
I20260807 03:03:51.175279 21113 raft_consensus.cc:2454] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04426a113741464cbd3c7a3eed158a01 in term 0.
I20260807 03:03:51.175432 21326 raft_consensus.cc:2454] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04426a113741464cbd3c7a3eed158a01 in term 0.
I20260807 03:03:51.175935 20794 leader_election.cc:302] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04426a113741464cbd3c7a3eed158a01, 7badc2b502bc46d29a0a02f4f274bb7e; no voters:
I20260807 03:03:51.176317 21435 raft_consensus.cc:2781] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:03:51.176452 21435 raft_consensus.cc:482] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:03:51.176560 21435 raft_consensus.cc:3037] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:51.178773 21435 raft_consensus.cc:504] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.179330 21435 leader_election.cc:288] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [CANDIDATE]: Term 1 election: Requested vote from peers 7badc2b502bc46d29a0a02f4f274bb7e (127.5.122.2:38449), c63b873446b045559ca45af41583d2fb (127.5.122.3:43081)
I20260807 03:03:51.179790 21113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3b901d4e14cf3b94b50878ea6e117" candidate_uuid: "04426a113741464cbd3c7a3eed158a01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7badc2b502bc46d29a0a02f4f274bb7e"
I20260807 03:03:51.179798 21326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3b901d4e14cf3b94b50878ea6e117" candidate_uuid: "04426a113741464cbd3c7a3eed158a01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c63b873446b045559ca45af41583d2fb"
I20260807 03:03:51.179960 21326 raft_consensus.cc:3037] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:51.179975 21113 raft_consensus.cc:3037] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:03:51.182135 21113 raft_consensus.cc:2454] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04426a113741464cbd3c7a3eed158a01 in term 1.
I20260807 03:03:51.182169 21326 raft_consensus.cc:2454] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04426a113741464cbd3c7a3eed158a01 in term 1.
I20260807 03:03:51.182525 20794 leader_election.cc:302] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04426a113741464cbd3c7a3eed158a01, 7badc2b502bc46d29a0a02f4f274bb7e; no voters:
I20260807 03:03:51.182812 21435 raft_consensus.cc:2781] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:03:51.183238 21435 raft_consensus.cc:686] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 1 LEADER]: Becoming Leader. State: Replica: 04426a113741464cbd3c7a3eed158a01, State: Running, Role: LEADER
I20260807 03:03:51.183712 21435 consensus_queue.cc:237] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } }
I20260807 03:03:51.187994 20670 catalog_manager.cc:5495] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04426a113741464cbd3c7a3eed158a01 (127.5.122.1). New cstate: current_term: 1 leader_uuid: "04426a113741464cbd3c7a3eed158a01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04426a113741464cbd3c7a3eed158a01" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36835 } health_report { overall_health: HEALTHY } } }
I20260807 03:03:51.206806 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.201667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.5.122.1:36835)
0807 03:03:51.201854 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:51.201861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:51.201919 (+ 58us) server_negotiation.cc:407] Connection header received
0807 03:03:51.202128 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:51.202132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:51.202204 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:51.202398 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:03:51.202753 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.203447 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.204765 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.205154 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.205259 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.205738 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:03:51.206215 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:03:51.206387 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:51.206467 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 03:03:51.206547 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:03:51.214650 21326 raft_consensus.cc:1261] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 1 FOLLOWER]: Refusing update from remote peer 04426a113741464cbd3c7a3eed158a01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:51.214650 21113 raft_consensus.cc:1261] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 1 FOLLOWER]: Refusing update from remote peer 04426a113741464cbd3c7a3eed158a01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:03:51.215693 21435 consensus_queue.cc:1035] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7badc2b502bc46d29a0a02f4f274bb7e" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:51.215987 21441 consensus_queue.cc:1035] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63b873446b045559ca45af41583d2fb" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:03:51.242432 20670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:51.242666 20670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:03:51.246052 20670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1ee576522914b2888f6ea87a3fa14fa: Sending DeleteTablet for 3 replicas of tablet a2c3b901d4e14cf3b94b50878ea6e117
I20260807 03:03:51.247345 20863 tablet_service.cc:1514] Processing DeleteTablet for tablet a2c3b901d4e14cf3b94b50878ea6e117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:51 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59122
I20260807 03:03:51.247421 21079 tablet_service.cc:1514] Processing DeleteTablet for tablet a2c3b901d4e14cf3b94b50878ea6e117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:51 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:60022
I20260807 03:03:51.248142 21292 tablet_service.cc:1514] Processing DeleteTablet for tablet a2c3b901d4e14cf3b94b50878ea6e117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:03:51 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:51296
I20260807 03:03:51.248313 21464 tablet_replica.cc:330] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: stopping tablet replica
I20260807 03:03:51.248852 21464 raft_consensus.cc:2229] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:03:51.249104 21467 tablet_replica.cc:330] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: stopping tablet replica
I20260807 03:03:51.249125 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 20768
I20260807 03:03:51.249493 21467 raft_consensus.cc:2229] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:51.249977 21467 raft_consensus.cc:2258] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:51.250239 21464 raft_consensus.cc:2258] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:51.253368 21464 ts_tablet_manager.cc:1905] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:51.261327 21464 ts_tablet_manager.cc:1918] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:51.261579 21464 log.cc:1196] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/a2c3b901d4e14cf3b94b50878ea6e117
I20260807 03:03:51.262217 21464 ts_tablet_manager.cc:1939] T a2c3b901d4e14cf3b94b50878ea6e117 P 04426a113741464cbd3c7a3eed158a01: Deleting consensus metadata
I20260807 03:03:51.263648 21468 tablet_replica.cc:330] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: stopping tablet replica
I20260807 03:03:51.264039 21468 raft_consensus.cc:2229] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:03:51.264526 21468 raft_consensus.cc:2258] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:03:51.264645 20651 catalog_manager.cc:4841] TS 04426a113741464cbd3c7a3eed158a01 (127.5.122.1:36835): tablet a2c3b901d4e14cf3b94b50878ea6e117 (table test-table [id=e7b45545e4ee4bd484a11209019c2180]) successfully deleted
I20260807 03:03:51.266428 21468 ts_tablet_manager.cc:1905] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:03:51.266559 21467 ts_tablet_manager.cc:1905] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 03:03:51 dist-test-slave-fgb3 krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:03:51.273242 21468 ts_tablet_manager.cc:1918] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:51.273438 21468 log.cc:1196] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/a2c3b901d4e14cf3b94b50878ea6e117
I20260807 03:03:51.273873 21468 ts_tablet_manager.cc:1939] T a2c3b901d4e14cf3b94b50878ea6e117 P c63b873446b045559ca45af41583d2fb: Deleting consensus metadata
I20260807 03:03:51.275636 20651 catalog_manager.cc:4841] TS c63b873446b045559ca45af41583d2fb (127.5.122.3:43081): tablet a2c3b901d4e14cf3b94b50878ea6e117 (table test-table [id=e7b45545e4ee4bd484a11209019c2180]) successfully deleted
I20260807 03:03:51.279170 21467 ts_tablet_manager.cc:1918] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:03:51.279325 21467 log.cc:1196] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/a2c3b901d4e14cf3b94b50878ea6e117
I20260807 03:03:51.280303 21467 ts_tablet_manager.cc:1939] T a2c3b901d4e14cf3b94b50878ea6e117 P 7badc2b502bc46d29a0a02f4f274bb7e: Deleting consensus metadata
I20260807 03:03:51.280323 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.250782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.5.122.1:36835)
0807 03:03:51.251021 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:51.251028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:51.260921 (+ 9893us) server_negotiation.cc:407] Connection header received
0807 03:03:51.261405 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:51.261410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:51.261556 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:51.261641 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:51.263408 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.264402 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.266597 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.267183 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.267282 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.272371 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:51.272387 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:51.272404 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:51.272450 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:51.276773 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:51.277594 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:51.277604 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:51.277608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:51.277785 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:51.278305 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:51.278333 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:51.278337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:51.278783 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:51.279098 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:51.279683 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 03:03:51.279942 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 03:03:51.282196 20648 catalog_manager.cc:4841] TS 7badc2b502bc46d29a0a02f4f274bb7e (127.5.122.2:38449): tablet a2c3b901d4e14cf3b94b50878ea6e117 (table test-table [id=e7b45545e4ee4bd484a11209019c2180]) successfully deleted
I20260807 03:03:51.282686 20958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:51.391088 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 20768
I20260807 03:03:51.413242 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 20977
Aug 07 03:03:51 dist-test-slave-fgb3 krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:03:51.427079 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.413977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.5.122.2:38449)
0807 03:03:51.414147 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:51.414154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:51.414233 (+ 79us) server_negotiation.cc:407] Connection header received
0807 03:03:51.414461 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:51.414466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:51.414543 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:51.414629 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:51.415848 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.416572 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.417778 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.418167 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.418477 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.422141 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:51.422155 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:51.422171 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:51.422207 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:51.424777 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:51.425341 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:51.425348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:51.425350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:51.425465 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:51.425716 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:51.425719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:51.425721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:51.426083 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:51.426365 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:51.426591 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 03:03:51.426750 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 03:03:51.427670 21178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:51.514935 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 20977
I20260807 03:03:51.535881 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 21190
Aug 07 03:03:51 dist-test-slave-fgb3 krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:03:51.550587 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:03:51.536708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.5.122.3:43081)
0807 03:03:51.536914 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:03:51.536919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:03:51.536940 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:03:51.537080 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:03:51.537084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:03:51.537155 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:03:51.537239 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:03:51.538634 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.539355 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.540583 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:03:51.540965 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:03:51.541073 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:03:51.545295 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:03:51.545312 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:03:51.545328 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:03:51.545369 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:03:51.547944 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:51.548580 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:51.548588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:51.548590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:51.548726 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:03:51.549076 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:03:51.549080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:03:51.549082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:03:51.549453 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:03:51.549734 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 03:03:51.550025 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 03:03:51.550235 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 03:03:51.551214 21390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:51.638707 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 21190
I20260807 03:03:51.658156 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 20615
I20260807 03:03:51.659303 20713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:03:51.765800 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 20615
2026-08-07T03:03:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4328 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:03:51 dist-test-slave-fgb3 krb5kdc[21480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:03:51 dist-test-slave-fgb3 krb5kdc[21480](info): set up 2 sockets
Aug 07 03:03:51 dist-test-slave-fgb3 krb5kdc[21480](info): commencing operation
krb5kdc: starting...
W20260807 03:03:52.946966 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:03:53 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:03:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:03:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/rangeradmin_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/rangeradmin_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/rangerlookup_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/rangerlookup_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
I20260807 03:03:53.304451 5608 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIpVSLO/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/postgres -l logfile start
2026-08-07 03:03:55.498 UTC [21530] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 03:03:55.498 UTC [21530] LOG: listening on IPv4 address "127.5.122.20", port 54601
2026-08-07 03:03:55.501 UTC [21530] LOG: listening on Unix socket "/tmp/.s.PGSQL.54601"
2026-08-07 03:03:55.522 UTC [21533] LOG: database system was shut down at 2026-08-07 03:03:54 UTC
2026-08-07 03:03:55.525 UTC [21530] LOG: database system is ready to accept connections
I20260807 03:03:56.600862 5608 mini_postgres.cc:96] Postgres bound to 54601
2026-08-07 03:03:56.623 UTC [21545] FATAL: database "slave" does not exist
127.5.122.20:54601 - accepting connections
I20260807 03:03:56.624229 5608 mini_ranger.cc:161] Starting Ranger...
I20260807 03:03:56.650264 5608 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 03:03:57.041975 5608 mini_ranger.cc:87] Created ranger Postgres database
I20260807 03:03:57.042115 5608 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-admin
2026-08-07 03:03:57,379 [I] DB FLAVOR :POSTGRES
2026-08-07 03:03:57,379 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 03:03:57,379 [I] Checking connection..
2026-08-07 03:03:57,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 03:03:57,639 [I] Checking connection passed.
2026-08-07 03:03:57,639 [I] --------- Verifying version history table ---------
2026-08-07 03:03:57,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 03:03:57,872 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 03:03:57,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 03:03:58,122 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 03:03:58,122 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 03:03:58,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 03:03:58.372 UTC [21647] WARNING: there is no transaction in progress
2026-08-07 03:03:58,379 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 03:03:58,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 03:03:58,635 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 03:03:58,636 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 03:03:58,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 03:03:58,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 03:03:59,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fgb3.c.gcp-upstream.internal','N') ;"
2026-08-07 03:03:59,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 03:03:59,624 [I] Table x_portal_user does not exist in database ranger
2026-08-07 03:03:59,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 03:03:59,879 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 03:03:59,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 03:04:00,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 03:04:00,378 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 03:04:00,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 03:04:01.240 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.255 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.269 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.282 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.295 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.310 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.360 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.366 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.379 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.388 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.397 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.409 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.418 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.423 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.429 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01.434 UTC [21820] WARNING: there is no transaction in progress
2026-08-07 03:04:01,959 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 03:04:01,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 03:04:02,205 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 03:04:02,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 03:04:02,456 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 03:04:02,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 03:04:02,724 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 03:04:02,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 03:04:02,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 03:04:03,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fgb3.c.gcp-upstream.internal';"
2026-08-07 03:04:03,462 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 03:04:03,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 03:04:03,697 [I] DB_PATCHES have already been applied
I20260807 03:04:03.703006 5608 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIpVSLO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 03:04:03.703094 5608 mini_ranger.cc:193] Using host: 127.5.122.20
I20260807 03:04:03.723479 5608 mini_ranger.cc:227] Ranger admin URL: http://127.5.122.20:39419
Aug 07, 2026 3:04:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:04:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:04:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:04:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:04:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:04:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:04:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 03:04:04.970887 5608 mini_ranger.cc:160] Time spent starting Ranger: real 8.347s user 0.003s sys 0.051s
Aug 07, 2026 3:04:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:04:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 3:04:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:04:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:04:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 03:04:27.288466 5608 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/rangerkms_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/rangerkms_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.122.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 03:04:27 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071867, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 03:04:27.475131 5608 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 03:04:27.498150 5608 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 03:04:28.238238 5608 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 03:04:28.238425 5608 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms
I20260807 03:04:28.238554 5608 mini_ranger_kms.cc:227] Using postgres at 127.5.122.20:54601
2026-08-07 03:04:28,880 [I] DB FLAVOR :POSTGRES
2026-08-07 03:04:28,880 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 03:04:28,881 [I] Checking connection
2026-08-07 03:04:28,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 03:04:29,136 [I] connection success
2026-08-07 03:04:29,136 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 03:04:29,136 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 03:04:29,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 03:04:29,372 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 03:04:29,372 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 03:04:29,372 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 03:04:29,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:54601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 03:04:29,627 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 03:04:30.090092 5608 mini_ranger_kms.cc:310] Created kms service
I20260807 03:04:30.298761 5608 mini_ranger_kms.cc:326] Created kudu user
I20260807 03:04:30.341315 5608 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 03:04:30 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071867, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:04:30.752800 5608 mini_ranger_kms.cc:384] Added ranger policy
I20260807 03:04:30.754376 5608 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 03:04:30.754441 5608 mini_ranger_kms.cc:240] Using host: 127.5.122.20
I20260807 03:04:30.778362 5608 mini_ranger_kms.cc:276] Ranger KMS PID: 22110
I20260807 03:04:30.778563 5608 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.122.20:52555
Aug 07, 2026 3:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:04:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:04:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 03:04:32 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786071872, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:04:33.235661 5608 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.761s user 0.135s sys 0.381s
I20260807 03:04:33.236052 5608 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 03:04:33.236321 5608 mini_ranger_kms.cc:401] 127.5.122.20:52555/kms/v1/keys
Aug 07 03:04:33 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071872, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.20@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
Aug 07 03:04:33 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071872, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.20@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
Aug 07 03:04:36 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:04:36.107365 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33989
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33989
--ranger_config_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:52555/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:04:36.549188 22196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:04:36.549485 22196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:04:36.549659 22196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:04:36.557754 22196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:04:36.557858 22196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:04:36.557933 22196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:04:36.558079 22196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:04:36.558148 22196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:04:36.569399 22196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.122.20:52555/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33989
--ranger_config_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:04:36.570747 22196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:04:36.572067 22196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:04:36.585562 22202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:04:36.585541 22203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:04:36.585879 22209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:04:36.587576 22196 server_base.cc:1034] running on GCE node
I20260807 03:04:36.588522 22196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:04:36.590299 22196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:04:36.591596 22196 hybrid_clock.cc:648] HybridClock initialized: now 1786071876591519 us; error 37 us; skew 500 ppm
I20260807 03:04:36.592079 22196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:04:36 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071876, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:04:36.599815 22196 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:04:36.601838 22196 webserver.cc:460] Webserver started at http://127.5.122.62:33659/ using document root <none> and password file <none>
I20260807 03:04:36.602833 22196 fs_manager.cc:359] Metadata directory not provided
I20260807 03:04:36.603009 22196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:04:36.603374 22196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:04:36 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071876, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:04:36.668758 22196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "57110be000da474987609e4744fc4084"
format_stamp: "Formatted at 2026-08-07 03:04:36 on dist-test-slave-fgb3"
server_key: "66ce613122cba8ffc851d6c027e2468b"
server_key_iv: "535646bfd3510d0dc006d7963e78678b"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:36.669648 22196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "57110be000da474987609e4744fc4084"
format_stamp: "Formatted at 2026-08-07 03:04:36 on dist-test-slave-fgb3"
server_key: "66ce613122cba8ffc851d6c027e2468b"
server_key_iv: "535646bfd3510d0dc006d7963e78678b"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:36.675622 22196 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260807 03:04:36.693678 22221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:36.695247 22196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:04:36.695456 22196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "57110be000da474987609e4744fc4084"
format_stamp: "Formatted at 2026-08-07 03:04:36 on dist-test-slave-fgb3"
server_key: "66ce613122cba8ffc851d6c027e2468b"
server_key_iv: "535646bfd3510d0dc006d7963e78678b"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:36.695708 22196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:04:36.714270 22196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:04:36.718830 22196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:04:36.719153 22196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:04:36.748672 22196 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:33989
I20260807 03:04:36.748845 22297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:33989 every 8 connection(s)
I20260807 03:04:36.749923 22196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:04:36.752408 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 22196
I20260807 03:04:36.752588 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:04:36.753202 5608 external_mini_cluster.cc:1428] Setting key 4ce44b1b08e182d5e27bfcea0dc86ca1
I20260807 03:04:36.753685 22299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 03:04:36 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:04:36.769595 22299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084: Bootstrap starting.
I20260807 03:04:36.773591 22299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084: Neither blocks nor log segments found. Creating new log.
I20260807 03:04:36.775127 22299 log.cc:824] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084: Log is configured to *not* fsync() on all Append() calls
I20260807 03:04:36.777693 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:36.755153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.5.122.62:33989)
0807 03:04:36.755900 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:36.755911 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:36.755952 (+ 41us) server_negotiation.cc:407] Connection header received
0807 03:04:36.758958 (+ 3006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:36.758981 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:36.759383 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:36.759704 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:36.761290 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:36.762911 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:36.764581 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:36.765335 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:36.765468 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:36.770151 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:36.770160 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:36.770172 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:36.770206 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:36.773380 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:36.774169 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:36.774175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:36.774181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:36.774439 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:36.774857 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:36.774860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:36.774862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:36.775476 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:36.775804 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:36.776234 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 03:04:36.776648 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":347,"threads_started":1}
I20260807 03:04:36.778244 22299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084: No bootstrap required, opened a new log
I20260807 03:04:36.787137 22299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57110be000da474987609e4744fc4084" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33989 } }
I20260807 03:04:36.787532 22299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:04:36.787638 22299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57110be000da474987609e4744fc4084, State: Initialized, Role: FOLLOWER
I20260807 03:04:36.788226 22299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57110be000da474987609e4744fc4084" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33989 } }
I20260807 03:04:36.788483 22299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:04:36.788623 22299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:04:36.788796 22299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:04:36.791491 22299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57110be000da474987609e4744fc4084" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33989 } }
I20260807 03:04:36.791982 22299 leader_election.cc:302] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57110be000da474987609e4744fc4084; no voters:
I20260807 03:04:36.792758 22299 leader_election.cc:288] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:04:36.792943 22307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:04:36.793850 22307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [term 1 LEADER]: Becoming Leader. State: Replica: 57110be000da474987609e4744fc4084, State: Running, Role: LEADER
I20260807 03:04:36.794428 22307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57110be000da474987609e4744fc4084" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33989 } }
I20260807 03:04:36.795028 22299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:04:36.797384 22308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57110be000da474987609e4744fc4084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57110be000da474987609e4744fc4084" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33989 } } }
I20260807 03:04:36.797693 22308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [sys.catalog]: This master's current role is: LEADER
I20260807 03:04:36.803438 22309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57110be000da474987609e4744fc4084. Latest consensus state: current_term: 1 leader_uuid: "57110be000da474987609e4744fc4084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57110be000da474987609e4744fc4084" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33989 } } }
I20260807 03:04:36.803959 22309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084 [sys.catalog]: This master's current role is: LEADER
I20260807 03:04:36.807019 22313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:04:36.807327 22299 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 03:04:36.819708 22313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:04:36.833843 22313 catalog_manager.cc:1347] Generated new cluster ID: 337fdeb9fdf041dd95d2ce09cdf17ae2
I20260807 03:04:36.834082 22313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:04:36.842514 22313 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:04:36.844019 22313 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:04:36.858882 22313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084: Generated new TSK 0
I20260807 03:04:36.860019 22313 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:04:37.722823 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:52555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33989
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:04:37 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786071877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 03:04:38.202142 22364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:04:38.202566 22364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:04:38.202749 22364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:04:38.216670 22364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:04:38.216905 22364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:04:38.217168 22364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:04:38.231776 22364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.122.20:52555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:04:38.233263 22364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:04:38.234728 22364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:04:38.257813 22380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:04:38.266655 22364 server_base.cc:1034] running on GCE node
W20260807 03:04:38.266758 22378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:04:38.267552 22385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:04:38.268325 22364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:04:38.269459 22364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:04:38.270712 22364 hybrid_clock.cc:648] HybridClock initialized: now 1786071878270625 us; error 56 us; skew 500 ppm
I20260807 03:04:38.271108 22364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:04:38 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:04:38.275312 22364 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:04:38.277411 22364 webserver.cc:460] Webserver started at http://127.5.122.1:44869/ using document root <none> and password file <none>
I20260807 03:04:38.278415 22364 fs_manager.cc:359] Metadata directory not provided
I20260807 03:04:38.278604 22364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:04:38.278999 22364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:04:38 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:04:38.299379 22364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "25ab024fce874660b5351699d10d356e"
format_stamp: "Formatted at 2026-08-07 03:04:38 on dist-test-slave-fgb3"
server_key: "d10b6dd2be360bae9e6f17b8b093a351"
server_key_iv: "f646b3485128ee45b9f3d70caa3ff339"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:38.300750 22364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "25ab024fce874660b5351699d10d356e"
format_stamp: "Formatted at 2026-08-07 03:04:38 on dist-test-slave-fgb3"
server_key: "d10b6dd2be360bae9e6f17b8b093a351"
server_key_iv: "f646b3485128ee45b9f3d70caa3ff339"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:38.309569 22364 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260807 03:04:38.352692 22393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:38.354988 22364 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.009s sys 0.008s
I20260807 03:04:38.355208 22364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "25ab024fce874660b5351699d10d356e"
format_stamp: "Formatted at 2026-08-07 03:04:38 on dist-test-slave-fgb3"
server_key: "d10b6dd2be360bae9e6f17b8b093a351"
server_key_iv: "f646b3485128ee45b9f3d70caa3ff339"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:38.355500 22364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:04:38.392458 22364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:04:38.398087 22364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:04:38.398524 22364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:04:38.410696 22364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:04:38.419211 22364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:04:38.419384 22364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:38.419574 22364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:04:38.419663 22364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:04:38 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:04:38.649025 22364 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:41149
I20260807 03:04:38.649065 22566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:41149 every 8 connection(s)
I20260807 03:04:38.650648 22364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:04:38.655305 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 22364
I20260807 03:04:38.655529 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:04:38.656435 5608 external_mini_cluster.cc:1428] Setting key fb2147f8941c2184b4453d929ab9897b
Aug 07 03:04:38 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071878, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:04:38.682138 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:38.654693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33989 (local address 127.5.122.1:47627)
0807 03:04:38.655955 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:38.656008 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:38.658544 (+ 2536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:38.659375 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:38.659388 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:38.660183 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:04:38.661050 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:38.661077 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:38.662845 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:38.662850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:38.665004 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:38.665019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:38.667124 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:38.667134 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:38.667501 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:38.668503 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:04:38.668538 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:04:38.672083 (+ 3545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:04:38.677111 (+ 5028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:04:38.677119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:04:38.677136 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:04:38.677775 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:04:38.678335 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:04:38.678341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:04:38.678344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:04:38.678699 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:04:38.679373 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:04:38.679381 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:04:38.679806 (+ 425us) client_negotiation.cc:769] Sending connection context
0807 03:04:38.680087 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 03:04:38.680659 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"thread_start_us":568,"threads_started":1}
I20260807 03:04:38.683825 22569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:38.653951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:47627 (local address 127.5.122.62:33989)
0807 03:04:38.654605 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:38.654609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:38.656148 (+ 1539us) server_negotiation.cc:407] Connection header received
0807 03:04:38.658847 (+ 2699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:38.658854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:38.658972 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:38.659105 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:38.661258 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:38.662648 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:38.665161 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:38.666605 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:38.666758 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:38.672415 (+ 5657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:38.672422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:38.672426 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:38.672466 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:38.676941 (+ 4475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:38.677932 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:38.677936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:38.677940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:38.678162 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:38.678848 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:38.678852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:38.678855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:38.679217 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:38.683002 (+ 3785us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:38.683148 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 03:04:38.683388 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":419,"threads_started":1}
I20260807 03:04:38.685374 22567 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33989
I20260807 03:04:38.685683 22567 heartbeater.cc:464] Registering TS with master...
I20260807 03:04:38.686437 22567 heartbeater.cc:511] Master 127.5.122.62:33989 requested a full tablet report, sending...
I20260807 03:04:38.688843 22242 ts_manager.cc:194] Registered new tserver with Master: 25ab024fce874660b5351699d10d356e (127.5.122.1:41149)
I20260807 03:04:38.690785 22242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:47627
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:04:38.737442 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:52555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33989
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:04:38 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071877, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
W20260807 03:04:39.318492 22575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:04:39.318768 22575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:04:39.318893 22575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:04:39.327577 22575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:04:39.327706 22575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:04:39.327886 22575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:04:39.339226 22575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.122.20:52555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:04:39.340451 22575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:04:39.341885 22575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:04:39.358975 22606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:04:39.360095 22609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:04:39.360306 22605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:04:39.361297 22575 server_base.cc:1034] running on GCE node
I20260807 03:04:39.362231 22575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:04:39.363474 22575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:04:39.364729 22575 hybrid_clock.cc:648] HybridClock initialized: now 1786071879364613 us; error 75 us; skew 500 ppm
I20260807 03:04:39.365211 22575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:04:39 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071879, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:04:39.369247 22575 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:04:39.371234 22575 webserver.cc:460] Webserver started at http://127.5.122.2:45189/ using document root <none> and password file <none>
I20260807 03:04:39.372253 22575 fs_manager.cc:359] Metadata directory not provided
I20260807 03:04:39.372452 22575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:04:39.372915 22575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:04:39 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071879, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:04:39.391844 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "21c5523ed2c849cda4f82a85c45d5b58"
format_stamp: "Formatted at 2026-08-07 03:04:39 on dist-test-slave-fgb3"
server_key: "7ed3c80d892cf52d1402d4829750da7d"
server_key_iv: "e4594e1613650e196ddea0814e34c437"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:39.393173 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "21c5523ed2c849cda4f82a85c45d5b58"
format_stamp: "Formatted at 2026-08-07 03:04:39 on dist-test-slave-fgb3"
server_key: "7ed3c80d892cf52d1402d4829750da7d"
server_key_iv: "e4594e1613650e196ddea0814e34c437"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:39.399876 22575 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 03:04:39.420221 22621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:39.422286 22575 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.002s
I20260807 03:04:39.422626 22575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "21c5523ed2c849cda4f82a85c45d5b58"
format_stamp: "Formatted at 2026-08-07 03:04:39 on dist-test-slave-fgb3"
server_key: "7ed3c80d892cf52d1402d4829750da7d"
server_key_iv: "e4594e1613650e196ddea0814e34c437"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:39.423067 22575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:04:39.447188 22575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:04:39.453322 22575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:04:39.453959 22575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:04:39.455529 22575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:04:39.457854 22575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:04:39.457978 22575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:39.458175 22575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:04:39.458266 22575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:39.515009 22575 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:33225
I20260807 03:04:39.515021 22786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:33225 every 8 connection(s)
I20260807 03:04:39.517134 22575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:04:39.521368 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 22575
I20260807 03:04:39.521548 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:04:39.522140 5608 external_mini_cluster.cc:1428] Setting key 54f9e227a306df073e28fea8bd7af057
Aug 07 03:04:39 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071879, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:04:39.540081 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:39.520073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:50555 (local address 127.5.122.62:33989)
0807 03:04:39.520826 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:39.520834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:39.522081 (+ 1247us) server_negotiation.cc:407] Connection header received
0807 03:04:39.525007 (+ 2926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:39.525013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:39.525108 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:39.525212 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:39.527024 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:39.527811 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:39.529821 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:39.530211 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:39.530388 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:39.534155 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:39.534160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:39.534163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:39.534201 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:39.536838 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:39.537589 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:39.537594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:39.537596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:39.537717 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:39.538276 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:39.538281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:39.538284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:39.538605 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:39.538751 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:39.539444 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 03:04:39.539684 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":458,"threads_started":1}
I20260807 03:04:39.541126 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:39.520752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33989 (local address 127.5.122.2:50555)
0807 03:04:39.521899 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:39.521954 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:39.524726 (+ 2772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:39.525487 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:39.525496 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:39.525993 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:04:39.526837 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:39.526858 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:39.527968 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:39.527973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:39.529685 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:39.529698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:39.530515 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:39.530523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:39.530835 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:39.531576 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:04:39.531598 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:04:39.533952 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:04:39.536985 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:04:39.536994 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:04:39.537012 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:04:39.537473 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:04:39.537821 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:04:39.537826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:04:39.537830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:04:39.538163 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:04:39.538751 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:04:39.538759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:04:39.539070 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 03:04:39.539328 (+ 258us) client_negotiation.cc:240] Negotiation successful
0807 03:04:39.539976 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":586,"threads_started":1}
I20260807 03:04:39.542573 22787 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33989
I20260807 03:04:39.542943 22787 heartbeater.cc:464] Registering TS with master...
I20260807 03:04:39.543993 22787 heartbeater.cc:511] Master 127.5.122.62:33989 requested a full tablet report, sending...
I20260807 03:04:39.545583 22242 ts_manager.cc:194] Registered new tserver with Master: 21c5523ed2c849cda4f82a85c45d5b58 (127.5.122.2:33225)
I20260807 03:04:39.546388 22242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:50555
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:04:39.579067 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:52555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33989
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:04:39.694301 22567 heartbeater.cc:503] Master 127.5.122.62:33989 was elected leader, sending a full tablet report...
W20260807 03:04:40.032997 22800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:04:40.033350 22800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:04:40.033514 22800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:04:40.042066 22800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:04:40.042204 22800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:04:40.042410 22800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:04:40.053503 22800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.122.20:52555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:04:40.054849 22800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:04:40.056126 22800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:04:40.070341 22807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:04:40.070482 22809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:04:40.070822 22806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:04:40.071712 22800 server_base.cc:1034] running on GCE node
I20260807 03:04:40.072348 22800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:04:40.073308 22800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:04:40.074496 22800 hybrid_clock.cc:648] HybridClock initialized: now 1786071880074433 us; error 49 us; skew 500 ppm
I20260807 03:04:40.074810 22800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:04:40 dist-test-slave-fgb3 krb5kdc[21480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071880, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:04:40.078246 22800 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:04:40.079955 22800 webserver.cc:460] Webserver started at http://127.5.122.3:34547/ using document root <none> and password file <none>
I20260807 03:04:40.080595 22800 fs_manager.cc:359] Metadata directory not provided
I20260807 03:04:40.080713 22800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:04:40.081051 22800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:04:40 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071880, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:04:40.097731 22800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "01ecc2248d524229bfe31f9d4c72a0a3"
format_stamp: "Formatted at 2026-08-07 03:04:40 on dist-test-slave-fgb3"
server_key: "bc9b2bc898b8d7a35b0f9550d369de5b"
server_key_iv: "ab86bc987f809762894e7cfc009e7cce"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:40.098729 22800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "01ecc2248d524229bfe31f9d4c72a0a3"
format_stamp: "Formatted at 2026-08-07 03:04:40 on dist-test-slave-fgb3"
server_key: "bc9b2bc898b8d7a35b0f9550d369de5b"
server_key_iv: "ab86bc987f809762894e7cfc009e7cce"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:40.104851 22800 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 03:04:40.120011 22818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:40.121730 22800 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:04:40.121937 22800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "01ecc2248d524229bfe31f9d4c72a0a3"
format_stamp: "Formatted at 2026-08-07 03:04:40 on dist-test-slave-fgb3"
server_key: "bc9b2bc898b8d7a35b0f9550d369de5b"
server_key_iv: "ab86bc987f809762894e7cfc009e7cce"
server_key_version: "kuduclusterkey@0"
I20260807 03:04:40.122212 22800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:04:40.177882 22800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:04:40.182273 22800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:04:40.182611 22800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:04:40.183696 22800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:04:40.185284 22800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:04:40.185387 22800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:40.185506 22800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:04:40.185550 22800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:04:40.235046 22800 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:45229
I20260807 03:04:40.235052 22990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:45229 every 8 connection(s)
I20260807 03:04:40.236963 22800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:04:40.247028 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 22800
I20260807 03:04:40.247197 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:04:40.247689 5608 external_mini_cluster.cc:1428] Setting key 96b101e2b292fd897125bf7af943f471
Aug 07 03:04:40 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071880, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:04:40.265215 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.241091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33989 (local address 127.5.122.3:53869)
0807 03:04:40.242460 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:40.242512 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:40.245651 (+ 3139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:40.246606 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:40.246617 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:40.247149 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:04:40.247935 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.247959 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.249254 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.249259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.251365 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.251378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.252486 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.252496 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.252921 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.253650 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:04:40.253683 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:04:40.256837 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:04:40.260654 (+ 3817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:04:40.260663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:04:40.260679 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:04:40.261329 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:04:40.261851 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:04:40.261857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:04:40.261861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:04:40.262216 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:04:40.262973 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:04:40.262981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:04:40.263437 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 03:04:40.263794 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 03:04:40.264224 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"thread_start_us":549,"threads_started":1}
I20260807 03:04:40.266366 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.240457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53869 (local address 127.5.122.62:33989)
0807 03:04:40.241204 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.241210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.242639 (+ 1429us) server_negotiation.cc:407] Connection header received
0807 03:04:40.245939 (+ 3300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.245943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.246058 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.246373 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:40.248127 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.249113 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.251520 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.252080 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.252178 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.257093 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:40.257098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:40.257101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:40.257132 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:40.260528 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.261472 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.261477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.261480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.261719 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.262370 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.262374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.262376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.262848 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:40.263020 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.265737 (+ 2717us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.265994 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":425,"threads_started":1}
I20260807 03:04:40.267474 22991 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33989
I20260807 03:04:40.267779 22991 heartbeater.cc:464] Registering TS with master...
I20260807 03:04:40.268488 22991 heartbeater.cc:511] Master 127.5.122.62:33989 requested a full tablet report, sending...
I20260807 03:04:40.269910 22242 ts_manager.cc:194] Registered new tserver with Master: 01ecc2248d524229bfe31f9d4c72a0a3 (127.5.122.3:45229)
I20260807 03:04:40.270924 22242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:53869
I20260807 03:04:40.276096 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:04:40.300232 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.281682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.5.122.62:33989)
0807 03:04:40.281864 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.281871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.282976 (+ 1105us) server_negotiation.cc:407] Connection header received
0807 03:04:40.283233 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.283239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.283334 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.283425 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:40.285523 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.286806 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.289529 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.290489 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.290649 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.294149 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:40.294158 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:40.294164 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:40.294208 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:40.297464 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.298285 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.298289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.298291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.298477 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.298879 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.298882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.298884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.299153 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:40.299332 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.299712 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.299993 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:04:40.304440 22242 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:04:40.312429 22242 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:04:40.348747 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.328436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35510 (local address 127.5.122.2:33225)
0807 03:04:40.329590 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.329599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.336864 (+ 7265us) server_negotiation.cc:407] Connection header received
0807 03:04:40.336971 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.336978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.337270 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.337510 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:04:40.340157 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.341976 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.345682 (+ 3706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.347168 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.347296 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.347536 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.348110 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.348321 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":1539,"threads_started":1}
I20260807 03:04:40.348958 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.328686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:33225 (local address 127.0.0.1:35510)
0807 03:04:40.336172 (+ 7486us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:40.336215 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:40.336666 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:40.338023 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:40.338031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:40.338066 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:04:40.339224 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.339240 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.342097 (+ 2857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.342102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.345484 (+ 3382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.345500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.347497 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.347505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.347799 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.347867 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 03:04:40.348044 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 03:04:40.348241 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7152,"mutex_wait_us":2,"thread_start_us":6788,"threads_started":1}
I20260807 03:04:40.349138 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.329759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.5.122.3:45229)
0807 03:04:40.336136 (+ 6377us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.336145 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.336866 (+ 721us) server_negotiation.cc:407] Connection header received
0807 03:04:40.336992 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.337001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.337264 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.337481 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:04:40.340325 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.342179 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.345741 (+ 3562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.347581 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.347730 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.347962 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.348453 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.348646 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6178,"thread_start_us":5978,"threads_started":1}
I20260807 03:04:40.349999 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.329875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:45229 (local address 127.0.0.1:43358)
0807 03:04:40.336567 (+ 6692us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:40.336583 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:40.336714 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:40.338174 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:40.338180 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:40.338209 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:04:40.340076 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.340091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.342362 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.342367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.345554 (+ 3187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.345566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.347866 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.347873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.348089 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.348121 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:04:40.348188 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 03:04:40.348302 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6550,"mutex_wait_us":19,"thread_start_us":6274,"threads_started":1}
I20260807 03:04:40.351702 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.336596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:41149 (local address 127.0.0.1:46112)
0807 03:04:40.340980 (+ 4384us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:40.340993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:40.341116 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:40.341689 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:40.341695 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:40.341720 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:04:40.342671 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.342688 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.345035 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.345040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.348064 (+ 3024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.348078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.350558 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.350566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.350825 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.350861 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 03:04:40.350942 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 03:04:40.351076 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4243,"thread_start_us":4063,"threads_started":1}
I20260807 03:04:40.352883 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.336947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.5.122.1:41149)
0807 03:04:40.340221 (+ 3274us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.340230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.341097 (+ 867us) server_negotiation.cc:407] Connection header received
0807 03:04:40.341301 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.341306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.341490 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.341732 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:04:40.342934 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.344888 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.348280 (+ 3392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.350237 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.351926 (+ 1689us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.352197 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.352282 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.352411 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3092,"thread_start_us":2809,"threads_started":1}
I20260807 03:04:40.354108 22693 tablet_service.cc:1467] Processing CreateTablet for tablet d248e368f70c41d389771186559c8c93 (DEFAULT_TABLE table=test-table [id=b42c4ee3504b41e78ca1e0b21ded344b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:04:40.354871 22889 tablet_service.cc:1467] Processing CreateTablet for tablet d248e368f70c41d389771186559c8c93 (DEFAULT_TABLE table=test-table [id=b42c4ee3504b41e78ca1e0b21ded344b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:04:40.355801 22693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d248e368f70c41d389771186559c8c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:04:40.356475 22889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d248e368f70c41d389771186559c8c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:04:40.361588 22470 tablet_service.cc:1467] Processing CreateTablet for tablet d248e368f70c41d389771186559c8c93 (DEFAULT_TABLE table=test-table [id=b42c4ee3504b41e78ca1e0b21ded344b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:04:40.362953 22470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d248e368f70c41d389771186559c8c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:04:40.374413 23017 tablet_bootstrap.cc:492] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Bootstrap starting.
I20260807 03:04:40.380049 23018 tablet_bootstrap.cc:492] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Bootstrap starting.
I20260807 03:04:40.380466 23017 tablet_bootstrap.cc:654] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Neither blocks nor log segments found. Creating new log.
I20260807 03:04:40.382709 23017 log.cc:824] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Log is configured to *not* fsync() on all Append() calls
I20260807 03:04:40.383446 23019 tablet_bootstrap.cc:492] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Bootstrap starting.
I20260807 03:04:40.384114 23018 tablet_bootstrap.cc:654] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Neither blocks nor log segments found. Creating new log.
I20260807 03:04:40.385902 23018 log.cc:824] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Log is configured to *not* fsync() on all Append() calls
I20260807 03:04:40.386739 23017 tablet_bootstrap.cc:492] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: No bootstrap required, opened a new log
I20260807 03:04:40.387110 23017 ts_tablet_manager.cc:1397] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.010s
I20260807 03:04:40.388599 23019 tablet_bootstrap.cc:654] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Neither blocks nor log segments found. Creating new log.
I20260807 03:04:40.390525 23019 log.cc:824] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Log is configured to *not* fsync() on all Append() calls
I20260807 03:04:40.393183 23018 tablet_bootstrap.cc:492] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: No bootstrap required, opened a new log
I20260807 03:04:40.393541 23018 ts_tablet_manager.cc:1397] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260807 03:04:40.396315 23019 tablet_bootstrap.cc:492] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: No bootstrap required, opened a new log
I20260807 03:04:40.396620 23019 ts_tablet_manager.cc:1397] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260807 03:04:40.397884 23017 raft_consensus.cc:348] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.398411 23017 raft_consensus.cc:374] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:04:40.398541 23017 raft_consensus.cc:729] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c5523ed2c849cda4f82a85c45d5b58, State: Initialized, Role: FOLLOWER
I20260807 03:04:40.399150 23017 consensus_queue.cc:260] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.400625 22787 heartbeater.cc:503] Master 127.5.122.62:33989 was elected leader, sending a full tablet report...
I20260807 03:04:40.401080 23017 ts_tablet_manager.cc:1428] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
I20260807 03:04:40.407800 23018 raft_consensus.cc:348] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.408250 23018 raft_consensus.cc:374] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:04:40.408391 23018 raft_consensus.cc:729] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ecc2248d524229bfe31f9d4c72a0a3, State: Initialized, Role: FOLLOWER
I20260807 03:04:40.409008 23018 consensus_queue.cc:260] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.409102 23019 raft_consensus.cc:348] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.409660 23019 raft_consensus.cc:374] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:04:40.409876 23019 raft_consensus.cc:729] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ab024fce874660b5351699d10d356e, State: Initialized, Role: FOLLOWER
I20260807 03:04:40.410450 22991 heartbeater.cc:503] Master 127.5.122.62:33989 was elected leader, sending a full tablet report...
I20260807 03:04:40.410894 23018 ts_tablet_manager.cc:1428] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Time spent starting tablet: real 0.017s user 0.006s sys 0.009s
I20260807 03:04:40.411381 23019 consensus_queue.cc:260] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.413208 23019 ts_tablet_manager.cc:1428] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Time spent starting tablet: real 0.016s user 0.005s sys 0.011s
W20260807 03:04:40.489486 22992 tablet.cc:2369] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:04:40.519819 22788 tablet.cc:2369] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:04:40.615486 23029 raft_consensus.cc:482] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:04:40.615973 23029 raft_consensus.cc:504] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.618136 23029 leader_election.cc:288] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c5523ed2c849cda4f82a85c45d5b58 (127.5.122.2:33225), 01ecc2248d524229bfe31f9d4c72a0a3 (127.5.122.3:45229)
I20260807 03:04:40.627810 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.618760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:33225 (local address 127.5.122.1:42505)
0807 03:04:40.620096 (+ 1336us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:40.620121 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:40.620378 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:40.620783 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:40.620787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:40.620811 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:04:40.621449 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.621465 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.622722 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.622727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.625016 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.625029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.626903 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.626913 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.627140 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.627177 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 03:04:40.627304 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 03:04:40.627435 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"thread_start_us":796,"threads_started":1}
I20260807 03:04:40.627911 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.619072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:42505 (local address 127.5.122.2:33225)
0807 03:04:40.619376 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.619382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.620258 (+ 876us) server_negotiation.cc:407] Connection header received
0807 03:04:40.620547 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.620552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.620632 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.620725 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:04:40.621604 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.622598 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.625169 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.626567 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.626726 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.626813 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.627330 (+ 517us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.627520 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 03:04:40.628521 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.619678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:45229 (local address 127.5.122.1:48361)
0807 03:04:40.620761 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0807 03:04:40.620785 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 03:04:40.620934 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:04:40.621249 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:04:40.621251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:04:40.621268 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:04:40.621916 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.621930 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.623430 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.623434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.625836 (+ 2402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:04:40.625852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.627515 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.627522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:04:40.627715 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.627746 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 03:04:40.627852 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 03:04:40.628063 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":1,"thread_start_us":744,"threads_started":1}
I20260807 03:04:40.628651 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.618614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:48361 (local address 127.5.122.3:45229)
0807 03:04:40.618865 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.618871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.620903 (+ 2032us) server_negotiation.cc:407] Connection header received
0807 03:04:40.621055 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.621060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.621134 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.621236 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:04:40.622100 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.623271 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.626008 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.627259 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.627414 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.627510 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.628163 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.628287 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 03:04:40.629356 22725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d248e368f70c41d389771186559c8c93" candidate_uuid: "25ab024fce874660b5351699d10d356e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c5523ed2c849cda4f82a85c45d5b58" is_pre_election: true
I20260807 03:04:40.630019 22921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d248e368f70c41d389771186559c8c93" candidate_uuid: "25ab024fce874660b5351699d10d356e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" is_pre_election: true
I20260807 03:04:40.630223 22725 raft_consensus.cc:2454] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25ab024fce874660b5351699d10d356e in term 0.
I20260807 03:04:40.630690 22921 raft_consensus.cc:2454] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25ab024fce874660b5351699d10d356e in term 0.
I20260807 03:04:40.631096 22398 leader_election.cc:302] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c5523ed2c849cda4f82a85c45d5b58, 25ab024fce874660b5351699d10d356e; no voters:
I20260807 03:04:40.631510 23029 raft_consensus.cc:2781] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:04:40.631707 23029 raft_consensus.cc:482] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:04:40.631873 23029 raft_consensus.cc:3037] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:04:40.634835 23029 raft_consensus.cc:504] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.635602 23029 leader_election.cc:288] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [CANDIDATE]: Term 1 election: Requested vote from peers 21c5523ed2c849cda4f82a85c45d5b58 (127.5.122.2:33225), 01ecc2248d524229bfe31f9d4c72a0a3 (127.5.122.3:45229)
I20260807 03:04:40.635998 22725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d248e368f70c41d389771186559c8c93" candidate_uuid: "25ab024fce874660b5351699d10d356e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c5523ed2c849cda4f82a85c45d5b58"
I20260807 03:04:40.636162 22725 raft_consensus.cc:3037] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:04:40.636132 22921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d248e368f70c41d389771186559c8c93" candidate_uuid: "25ab024fce874660b5351699d10d356e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ecc2248d524229bfe31f9d4c72a0a3"
I20260807 03:04:40.636348 22921 raft_consensus.cc:3037] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:04:40.639099 22725 raft_consensus.cc:2454] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25ab024fce874660b5351699d10d356e in term 1.
I20260807 03:04:40.639535 22398 leader_election.cc:302] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c5523ed2c849cda4f82a85c45d5b58, 25ab024fce874660b5351699d10d356e; no voters:
I20260807 03:04:40.639566 22921 raft_consensus.cc:2454] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25ab024fce874660b5351699d10d356e in term 1.
I20260807 03:04:40.639876 23029 raft_consensus.cc:2781] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:04:40.640547 23029 raft_consensus.cc:686] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 1 LEADER]: Becoming Leader. State: Replica: 25ab024fce874660b5351699d10d356e, State: Running, Role: LEADER
I20260807 03:04:40.641073 23029 consensus_queue.cc:237] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } }
I20260807 03:04:40.645748 22242 catalog_manager.cc:5495] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e reported cstate change: term changed from 0 to 1, leader changed from <none> to 25ab024fce874660b5351699d10d356e (127.5.122.1). New cstate: current_term: 1 leader_uuid: "25ab024fce874660b5351699d10d356e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25ab024fce874660b5351699d10d356e" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 41149 } health_report { overall_health: HEALTHY } } }
W20260807 03:04:40.654342 22568 tablet.cc:2369] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:04:40.705430 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.698556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46124 (local address 127.5.122.1:41149)
0807 03:04:40.698775 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.698781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.698801 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:04:40.699032 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.699036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.699130 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.699262 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:04:40.699738 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.700819 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.702921 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.703585 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.703701 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.704243 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:04:40.704765 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:04:40.704905 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.705012 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.705101 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 03:04:40.714977 22921 raft_consensus.cc:1261] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 1 FOLLOWER]: Refusing update from remote peer 25ab024fce874660b5351699d10d356e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:04:40.715163 22725 raft_consensus.cc:1261] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 1 FOLLOWER]: Refusing update from remote peer 25ab024fce874660b5351699d10d356e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:04:40.715958 23034 consensus_queue.cc:1035] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ecc2248d524229bfe31f9d4c72a0a3" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 45229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:04:40.716440 23029 consensus_queue.cc:1035] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c5523ed2c849cda4f82a85c45d5b58" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:04:40.731371 23041 mvcc.cc:205] Tried to move back new op lower bound from 7315750423398244352 to 7315750423109644288. Current Snapshot: MvccSnapshot[applied={T|T < 7315750423398244352}]
I20260807 03:04:40.733054 23039 mvcc.cc:205] Tried to move back new op lower bound from 7315750423398244352 to 7315750423109644288. Current Snapshot: MvccSnapshot[applied={T|T < 7315750423398244352 or (T in {7315750423398244352})}]
I20260807 03:04:40.743656 22240 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:04:40.744154 22240 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:04:40.749063 22240 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57110be000da474987609e4744fc4084: Sending DeleteTablet for 3 replicas of tablet d248e368f70c41d389771186559c8c93
I20260807 03:04:40.750545 22889 tablet_service.cc:1514] Processing DeleteTablet for tablet d248e368f70c41d389771186559c8c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:04:40 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:43358
I20260807 03:04:40.750692 22693 tablet_service.cc:1514] Processing DeleteTablet for tablet d248e368f70c41d389771186559c8c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:04:40 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:35510
I20260807 03:04:40.752532 22470 tablet_service.cc:1514] Processing DeleteTablet for tablet d248e368f70c41d389771186559c8c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:04:40 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:46112
I20260807 03:04:40.756493 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 22364
I20260807 03:04:40.763624 23054 tablet_replica.cc:330] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: stopping tablet replica
I20260807 03:04:40.764452 23054 raft_consensus.cc:2229] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:04:40.765453 23053 tablet_replica.cc:330] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: stopping tablet replica
I20260807 03:04:40.766077 23054 raft_consensus.cc:2258] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:04:40.766675 23053 raft_consensus.cc:2229] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:04:40.767725 23053 raft_consensus.cc:2258] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:04:40.771179 23056 tablet_replica.cc:330] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: stopping tablet replica
I20260807 03:04:40.771584 23053 ts_tablet_manager.cc:1905] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:04:40.771685 23056 raft_consensus.cc:2229] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:04:40.771979 23054 ts_tablet_manager.cc:1905] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:04:40.772542 23056 raft_consensus.cc:2258] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:04:40.777173 23056 ts_tablet_manager.cc:1905] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:04:40.781602 23054 ts_tablet_manager.cc:1918] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:04:40.784248 23056 ts_tablet_manager.cc:1918] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:04:40.784490 23056 log.cc:1196] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/d248e368f70c41d389771186559c8c93
I20260807 03:04:40.785132 23056 ts_tablet_manager.cc:1939] T d248e368f70c41d389771186559c8c93 P 25ab024fce874660b5351699d10d356e: Deleting consensus metadata
I20260807 03:04:40.786454 23054 log.cc:1196] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/d248e368f70c41d389771186559c8c93
I20260807 03:04:40.787070 22223 catalog_manager.cc:4841] TS 25ab024fce874660b5351699d10d356e (127.5.122.1:41149): tablet d248e368f70c41d389771186559c8c93 (table test-table [id=b42c4ee3504b41e78ca1e0b21ded344b]) successfully deleted
I20260807 03:04:40.787532 23054 ts_tablet_manager.cc:1939] T d248e368f70c41d389771186559c8c93 P 21c5523ed2c849cda4f82a85c45d5b58: Deleting consensus metadata
I20260807 03:04:40.789729 22223 catalog_manager.cc:4841] TS 21c5523ed2c849cda4f82a85c45d5b58 (127.5.122.2:33225): tablet d248e368f70c41d389771186559c8c93 (table test-table [id=b42c4ee3504b41e78ca1e0b21ded344b]) successfully deleted
Aug 07 03:04:40 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:04:40.791324 23053 ts_tablet_manager.cc:1918] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:04:40.791528 23053 log.cc:1196] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/d248e368f70c41d389771186559c8c93
I20260807 03:04:40.792059 23053 ts_tablet_manager.cc:1939] T d248e368f70c41d389771186559c8c93 P 01ecc2248d524229bfe31f9d4c72a0a3: Deleting consensus metadata
I20260807 03:04:40.793777 22224 catalog_manager.cc:4841] TS 01ecc2248d524229bfe31f9d4c72a0a3 (127.5.122.3:45229): tablet d248e368f70c41d389771186559c8c93 (table test-table [id=b42c4ee3504b41e78ca1e0b21ded344b]) successfully deleted
I20260807 03:04:40.798069 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.757898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.5.122.1:41149)
0807 03:04:40.758106 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.758113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.773406 (+ 15293us) server_negotiation.cc:407] Connection header received
0807 03:04:40.773641 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.773646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.773740 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.773866 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:40.775961 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.777104 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.783612 (+ 6508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.784298 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.784406 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.791708 (+ 7302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:40.791725 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:40.791744 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:40.791822 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:40.795069 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.795821 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.795828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.795830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.795987 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.796359 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.796363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.796382 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.796864 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:40.797169 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.797458 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.797690 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 03:04:40.798803 22560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:04:40.896821 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 22364
I20260807 03:04:40.917945 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 22575
Aug 07 03:04:40 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:04:40.933241 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:40.918723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35524 (local address 127.5.122.2:33225)
0807 03:04:40.918905 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:40.918910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:40.918953 (+ 43us) server_negotiation.cc:407] Connection header received
0807 03:04:40.919153 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:40.919157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:40.919219 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:40.919301 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:40.920707 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.921393 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.922791 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:40.923251 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:40.923369 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:40.927708 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:40.927721 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:40.927736 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:40.927780 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:40.930659 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.931275 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.931282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.931284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.931416 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:40.931747 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:40.931750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:40.931752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:40.932136 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:40.932400 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:40.932706 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 03:04:40.932921 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:04:40.933843 22781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:04:41.020885 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 22575
I20260807 03:04:41.042975 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 22800
Aug 07 03:04:41 dist-test-slave-fgb3 krb5kdc[21480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:04:41.057804 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:04:41.043725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.5.122.3:45229)
0807 03:04:41.043882 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 03:04:41.043887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:04:41.043947 (+ 60us) server_negotiation.cc:407] Connection header received
0807 03:04:41.044114 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:04:41.044118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:04:41.044187 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:04:41.044284 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:04:41.045721 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:41.046504 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:41.047811 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:04:41.048200 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:04:41.048322 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:04:41.052767 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:04:41.052783 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:04:41.052800 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:04:41.052833 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:04:41.055420 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:41.056071 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:41.056078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:41.056080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:41.056200 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:04:41.056506 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:04:41.056509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:04:41.056511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:04:41.056839 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:04:41.057065 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 03:04:41.057300 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 03:04:41.057459 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:04:41.058512 22986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:04:41.145152 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 22800
I20260807 03:04:41.164949 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 22196
I20260807 03:04:41.166147 22291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:04:41.276669 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 22196
I20260807 03:04:41.311960 5608 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 03:04:41.657127 5608 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 03:04:41.657433 5608 mini_ranger.cc:66] Stopping Ranger...
I20260807 03:04:42.001857 5608 mini_ranger.cc:68] Stopped Ranger
2026-08-07 03:04:42.002 UTC [21530] LOG: received smart shutdown request
2026-08-07 03:04:42.005 UTC [21530] LOG: background worker "logical replication launcher" (PID 21542) exited with exit code 1
2026-08-07 03:04:42.005 UTC [21534] LOG: shutting down
2026-08-07 03:04:42.128 UTC [21530] LOG: database system is shut down
2026-08-07T03:04:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50464 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:04:42 dist-test-slave-fgb3 krb5kdc[23078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:04:42 dist-test-slave-fgb3 krb5kdc[23078](info): set up 2 sockets
Aug 07 03:04:42 dist-test-slave-fgb3 krb5kdc[23078](info): commencing operation
krb5kdc: starting...
W20260807 03:04:43.435663 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:04:43 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:04:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:04:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/rangeradmin_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/rangeradmin_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/rangerlookup_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/rangerlookup_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
I20260807 03:04:43.758301 5608 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIpVSLO/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/postgres -l logfile start
2026-08-07 03:04:45.891 UTC [23124] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 03:04:45.891 UTC [23124] LOG: listening on IPv4 address "127.5.122.20", port 44271
2026-08-07 03:04:45.894 UTC [23124] LOG: listening on Unix socket "/tmp/.s.PGSQL.44271"
2026-08-07 03:04:45.914 UTC [23129] LOG: database system was shut down at 2026-08-07 03:04:44 UTC
2026-08-07 03:04:45.918 UTC [23124] LOG: database system is ready to accept connections
I20260807 03:04:46.994616 5608 mini_postgres.cc:96] Postgres bound to 44271
2026-08-07 03:04:47.019 UTC [23141] FATAL: database "slave" does not exist
127.5.122.20:44271 - accepting connections
I20260807 03:04:47.020334 5608 mini_ranger.cc:161] Starting Ranger...
I20260807 03:04:47.046636 5608 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 03:04:47.412962 5608 mini_ranger.cc:87] Created ranger Postgres database
I20260807 03:04:47.413077 5608 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-admin
2026-08-07 03:04:47,703 [I] DB FLAVOR :POSTGRES
2026-08-07 03:04:47,704 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 03:04:47,704 [I] Checking connection..
2026-08-07 03:04:47,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 03:04:47,954 [I] Checking connection passed.
2026-08-07 03:04:47,955 [I] --------- Verifying version history table ---------
2026-08-07 03:04:47,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 03:04:48,217 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 03:04:48,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 03:04:48,443 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 03:04:48,444 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 03:04:48,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 03:04:48.693 UTC [23236] WARNING: there is no transaction in progress
2026-08-07 03:04:48,701 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 03:04:48,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 03:04:48,937 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 03:04:48,937 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 03:04:48,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 03:04:49,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 03:04:49,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fgb3.c.gcp-upstream.internal','N') ;"
2026-08-07 03:04:49,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 03:04:49,937 [I] Table x_portal_user does not exist in database ranger
2026-08-07 03:04:49,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 03:04:50,193 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 03:04:50,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 03:04:50,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 03:04:50,672 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 03:04:50,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 03:04:51.531 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.545 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.559 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.572 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.585 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.600 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.649 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.655 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.667 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.676 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.684 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.697 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.705 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.710 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.715 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:51.720 UTC [23402] WARNING: there is no transaction in progress
2026-08-07 03:04:52,243 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 03:04:52,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 03:04:52,503 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 03:04:52,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 03:04:52,766 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 03:04:52,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 03:04:53,013 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 03:04:53,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 03:04:53,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 03:04:53,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fgb3.c.gcp-upstream.internal';"
2026-08-07 03:04:53,721 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 03:04:53,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 03:04:53,976 [I] DB_PATCHES have already been applied
I20260807 03:04:53.982672 5608 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIpVSLO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 03:04:53.982764 5608 mini_ranger.cc:193] Using host: 127.5.122.20
I20260807 03:04:54.000939 5608 mini_ranger.cc:227] Ranger admin URL: http://127.5.122.20:53431
Aug 07, 2026 3:04:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:04:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:04:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:04:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:04:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:04:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:04:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 03:04:55.217095 5608 mini_ranger.cc:160] Time spent starting Ranger: real 8.197s user 0.003s sys 0.048s
Aug 07, 2026 3:05:14 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:05:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 3:05:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:05:14 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:05:15 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 03:05:15.653683 5608 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.122.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.122.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/rangerkms_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.122.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/rangerkms_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.122.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/HTTP_127.5.122.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 03:05:15 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071915, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 03:05:15.848915 5608 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 03:05:15.872550 5608 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 03:05:16.592366 5608 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 03:05:16.592494 5608 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms
I20260807 03:05:16.592533 5608 mini_ranger_kms.cc:227] Using postgres at 127.5.122.20:44271
2026-08-07 03:05:16,958 [I] DB FLAVOR :POSTGRES
2026-08-07 03:05:16,959 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 03:05:16,959 [I] Checking connection
2026-08-07 03:05:16,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 03:05:17,232 [I] connection success
2026-08-07 03:05:17,232 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 03:05:17,232 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 03:05:17,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 03:05:17,503 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 03:05:17,503 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 03:05:17,504 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 03:05:17,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.20:44271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 03:05:17,789 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 03:05:18.256139 5608 mini_ranger_kms.cc:310] Created kms service
I20260807 03:05:18.436793 5608 mini_ranger_kms.cc:326] Created kudu user
I20260807 03:05:18.477675 5608 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 03:05:18 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071915, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:18.864493 5608 mini_ranger_kms.cc:384] Added ranger policy
I20260807 03:05:18.864912 5608 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIpVSLO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIpVSLO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 03:05:18.864990 5608 mini_ranger_kms.cc:240] Using host: 127.5.122.20
I20260807 03:05:18.886646 5608 mini_ranger_kms.cc:276] Ranger KMS PID: 23687
I20260807 03:05:18.886776 5608 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.122.20:34359
Aug 07, 2026 3:05:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:05:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:05:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:05:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 03:05:21 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786071921, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:21.306588 5608 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.458s user 0.071s sys 0.327s
I20260807 03:05:21.306874 5608 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 03:05:21.307046 5608 mini_ranger_kms.cc:401] 127.5.122.20:34359/kms/v1/keys
Aug 07 03:05:21 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071921, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.20@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
Aug 07 03:05:21 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071921, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.20@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
Aug 07 03:05:24 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:05:24.172366 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42265
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:42265
--ranger_config_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:24.499176 23769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:24.499455 23769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:24.499574 23769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:24.506680 23769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:24.506775 23769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:24.506824 23769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:24.506901 23769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:24.506960 23769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:24.517221 23769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.122.20:34359/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:42265
--ranger_config_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:24.518456 23769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:24.519806 23769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:24.531045 23778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:24.531025 23780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:24.531438 23783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:24.531663 23769 server_base.cc:1034] running on GCE node
I20260807 03:05:24.532410 23769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:24.533691 23769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:24.534894 23769 hybrid_clock.cc:648] HybridClock initialized: now 1786071924534842 us; error 42 us; skew 500 ppm
I20260807 03:05:24.535230 23769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:24 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:24.538229 23769 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:05:24.539580 23769 webserver.cc:460] Webserver started at http://127.5.122.62:37077/ using document root <none> and password file <none>
I20260807 03:05:24.540198 23769 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:24.540307 23769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:24.540589 23769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:05:24 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:24.588505 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc"
format_stamp: "Formatted at 2026-08-07 03:05:24 on dist-test-slave-fgb3"
server_key: "f8842b6f086b2e433f953a25a7df66e8"
server_key_iv: "2f8db1a2bea3f98ff832f64cad09706b"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:24.589340 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc"
format_stamp: "Formatted at 2026-08-07 03:05:24 on dist-test-slave-fgb3"
server_key: "f8842b6f086b2e433f953a25a7df66e8"
server_key_iv: "2f8db1a2bea3f98ff832f64cad09706b"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:24.595043 23769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 03:05:24.614569 23791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:24.615980 23769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:05:24.616139 23769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc"
format_stamp: "Formatted at 2026-08-07 03:05:24 on dist-test-slave-fgb3"
server_key: "f8842b6f086b2e433f953a25a7df66e8"
server_key_iv: "2f8db1a2bea3f98ff832f64cad09706b"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:24.616360 23769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:24.639609 23769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:24.643468 23769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:24.643772 23769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:24.661293 23769 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:42265
I20260807 03:05:24.661329 23874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:42265 every 8 connection(s)
I20260807 03:05:24.662652 23769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:05:24.666256 23875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:24.669512 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 23769
I20260807 03:05:24.669701 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:05:24.670404 5608 external_mini_cluster.cc:1428] Setting key d2ae01452241046915bf100f8df54cc2
I20260807 03:05:24.678723 23875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Bootstrap starting.
I20260807 03:05:24.682597 23875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:24.683895 23875 log.cc:824] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Log is configured to *not* fsync() on all Append() calls
Aug 07 03:05:24 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:24.686893 23875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: No bootstrap required, opened a new log
I20260807 03:05:24.692534 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:24.672344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.5.122.62:42265)
0807 03:05:24.673081 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:24.673092 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:24.673134 (+ 42us) server_negotiation.cc:407] Connection header received
0807 03:05:24.674748 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:24.674774 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:24.675205 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:24.675558 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:24.677137 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:24.678451 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:24.680173 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:24.680839 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:24.680973 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:24.685606 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:24.685614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:24.685628 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:24.685668 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:24.688574 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:24.689231 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:24.689238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:24.689244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:24.689432 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:24.689811 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:24.689815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:24.689817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:24.690375 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:24.690670 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:24.691116 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 03:05:24.691526 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":338,"threads_started":1}
I20260807 03:05:24.695029 23875 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:24.695441 23875 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:24.695560 23875 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8af6fbfd1d54bdda2efd19f78ca94bc, State: Initialized, Role: FOLLOWER
I20260807 03:05:24.696105 23875 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:24.696326 23875 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:05:24.696461 23875 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:05:24.696602 23875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:24.698735 23875 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:24.699186 23875 leader_election.cc:302] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8af6fbfd1d54bdda2efd19f78ca94bc; no voters:
I20260807 03:05:24.699719 23875 leader_election.cc:288] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:05:24.699863 23881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:05:24.700532 23881 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 LEADER]: Becoming Leader. State: Replica: c8af6fbfd1d54bdda2efd19f78ca94bc, State: Running, Role: LEADER
I20260807 03:05:24.701081 23881 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:24.701646 23875 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:24.703881 23882 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } } }
I20260807 03:05:24.703912 23883 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8af6fbfd1d54bdda2efd19f78ca94bc. Latest consensus state: current_term: 1 leader_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } } }
I20260807 03:05:24.704314 23882 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:24.704344 23883 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:24.705754 23888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:05:24.710817 23888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:05:24.716807 23875 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 03:05:24.727900 23888 catalog_manager.cc:1347] Generated new cluster ID: b7705f764ed841ffaf82875823aee88e
I20260807 03:05:24.728128 23888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:05:24.746767 23888 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:05:24.748647 23888 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:05:24.760361 23888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Generated new TSK 0
I20260807 03:05:24.761409 23888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:05:25.487304 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42265
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:05:25 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786071925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 03:05:25.901607 23950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:25.901965 23950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:25.902173 23950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:25.913476 23950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:25.913631 23950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:25.913837 23950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:05:25.929373 23950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.122.20:34359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:25.931411 23950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:25.933218 23950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:25.961500 23968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:25.962250 23967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:25.970585 23950 server_base.cc:1034] running on GCE node
W20260807 03:05:25.970952 23974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:25.971801 23950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:25.973271 23950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:25.974555 23950 hybrid_clock.cc:648] HybridClock initialized: now 1786071925974481 us; error 57 us; skew 500 ppm
I20260807 03:05:25.974980 23950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:25 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:25.979353 23950 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:05:25.983153 23950 webserver.cc:460] Webserver started at http://127.5.122.1:37451/ using document root <none> and password file <none>
I20260807 03:05:25.984088 23950 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:25.984229 23950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:25.984603 23950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:05:25 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:26.005363 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "aaa1729f451144d78ce59397b6875b37"
format_stamp: "Formatted at 2026-08-07 03:05:25 on dist-test-slave-fgb3"
server_key: "f724db8f791426a9e0de3ddfc95bd4be"
server_key_iv: "aa0b4bfc5acb1b615d8dc226b1334529"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:26.006618 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "aaa1729f451144d78ce59397b6875b37"
format_stamp: "Formatted at 2026-08-07 03:05:25 on dist-test-slave-fgb3"
server_key: "f724db8f791426a9e0de3ddfc95bd4be"
server_key_iv: "aa0b4bfc5acb1b615d8dc226b1334529"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:26.019783 23950 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.004s
I20260807 03:05:26.044143 23983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:26.046088 23950 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260807 03:05:26.046357 23950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "aaa1729f451144d78ce59397b6875b37"
format_stamp: "Formatted at 2026-08-07 03:05:25 on dist-test-slave-fgb3"
server_key: "f724db8f791426a9e0de3ddfc95bd4be"
server_key_iv: "aa0b4bfc5acb1b615d8dc226b1334529"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:26.046722 23950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:26.076556 23950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:26.081894 23950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:26.082273 23950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:26.083564 23950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:26.085414 23950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:26.085518 23950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:26.085656 23950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:26.085719 23950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:26.188962 23950 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:35427
I20260807 03:05:26.189020 24169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:35427 every 8 connection(s)
I20260807 03:05:26.190672 23950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:05:26.196974 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 23950
I20260807 03:05:26.197193 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:05:26.198097 5608 external_mini_cluster.cc:1428] Setting key dd0ef1a5533e0c83caf417f5e371fe94
Aug 07 03:05:26 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
I20260807 03:05:26.242347 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:26.194538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42265 (local address 127.5.122.1:51717)
0807 03:05:26.211037 (+ 16499us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:26.211102 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:26.213760 (+ 2658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:26.215828 (+ 2068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:26.215841 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:26.216494 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:26.217315 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:26.217343 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:26.222439 (+ 5096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:26.222445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:26.224454 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:26.224477 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:26.225748 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:26.225755 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:26.226054 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:26.226828 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:26.226856 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:26.232606 (+ 5750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:26.236614 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:26.236624 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:26.236643 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:26.237385 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:26.237960 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:26.237965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:26.237969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:26.238234 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:26.239635 (+ 1401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:26.239644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:26.240097 (+ 453us) client_negotiation.cc:769] Sending connection context
0807 03:05:26.240401 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 03:05:26.240959 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16119,"thread_start_us":15877,"threads_started":1}
I20260807 03:05:26.245220 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:26.193853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:51717 (local address 127.5.122.62:42265)
0807 03:05:26.214176 (+ 20323us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:26.214181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:26.214202 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:05:26.214529 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:26.214534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:26.214637 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:26.214747 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:26.217516 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:26.218705 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:26.224629 (+ 5924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:26.225292 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:26.225387 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:26.232909 (+ 7522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:26.232917 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:26.232922 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:26.232962 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:26.236430 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:26.237545 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:26.237551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:26.237554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:26.237716 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:26.239086 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:26.239092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:26.239095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:26.239486 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:26.239643 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:26.244598 (+ 4955us) server_negotiation.cc:299] Negotiation successful
0807 03:05:26.244838 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20205,"thread_start_us":20071,"threads_started":1}
I20260807 03:05:26.251036 24170 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42265
I20260807 03:05:26.251413 24170 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:26.252383 24170 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
I20260807 03:05:26.255136 23823 ts_manager.cc:194] Registered new tserver with Master: aaa1729f451144d78ce59397b6875b37 (127.5.122.1:35427)
I20260807 03:05:26.257203 23823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:51717
Aug 07 03:05:26 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:05:26.291733 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42265
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:05:26 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
W20260807 03:05:26.759569 24181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:26.760066 24181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:26.760258 24181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:26.771704 24181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:26.771920 24181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:26.772146 24181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:05:26.787634 24181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.122.20:34359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:26.789464 24181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:26.791177 24181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:26.823967 24194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:26.827978 24198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:26.828455 24181 server_base.cc:1034] running on GCE node
W20260807 03:05:26.828946 24193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:26.829834 24181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:26.836818 24181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:26.838363 24181 hybrid_clock.cc:648] HybridClock initialized: now 1786071926838250 us; error 107 us; skew 500 ppm
I20260807 03:05:26.838835 24181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:26 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:26.845896 24181 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:05:26.868088 24181 webserver.cc:460] Webserver started at http://127.5.122.2:44253/ using document root <none> and password file <none>
I20260807 03:05:26.869324 24181 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:26.869570 24181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:26.870026 24181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:05:26 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:26.915212 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "fd675f6727d54fa8bd54d30fb55fb3bf"
format_stamp: "Formatted at 2026-08-07 03:05:26 on dist-test-slave-fgb3"
server_key: "aa43f1b55b71db71b81ceda64dd66853"
server_key_iv: "df8c46c6f01b86394b92d008d9fef3db"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:26.927103 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "fd675f6727d54fa8bd54d30fb55fb3bf"
format_stamp: "Formatted at 2026-08-07 03:05:26 on dist-test-slave-fgb3"
server_key: "aa43f1b55b71db71b81ceda64dd66853"
server_key_iv: "df8c46c6f01b86394b92d008d9fef3db"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:26.940523 24181 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.000s
I20260807 03:05:26.968093 24213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:26.969941 24181 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.006s sys 0.005s
I20260807 03:05:26.970189 24181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "fd675f6727d54fa8bd54d30fb55fb3bf"
format_stamp: "Formatted at 2026-08-07 03:05:26 on dist-test-slave-fgb3"
server_key: "aa43f1b55b71db71b81ceda64dd66853"
server_key_iv: "df8c46c6f01b86394b92d008d9fef3db"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:26.970518 24181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:27.020764 24181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:27.025971 24181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:27.026371 24181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:27.033788 24181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:27.041822 24181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:27.041965 24181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:27.042124 24181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:27.042212 24181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:27.260928 24170 heartbeater.cc:503] Master 127.5.122.62:42265 was elected leader, sending a full tablet report...
I20260807 03:05:27.382057 24181 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:44285
I20260807 03:05:27.382071 24390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:44285 every 8 connection(s)
I20260807 03:05:27.383618 24181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:05:27.390681 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 24181
I20260807 03:05:27.390861 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:05:27.391418 5608 external_mini_cluster.cc:1428] Setting key 8069db9f715bf15b9236c78c67fc4279
Aug 07 03:05:27 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:27.403829 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:27.386379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:48703 (local address 127.5.122.62:42265)
0807 03:05:27.386827 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:27.386831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:27.387821 (+ 990us) server_negotiation.cc:407] Connection header received
0807 03:05:27.389593 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:27.389599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:27.389690 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:27.389786 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:27.391546 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.392632 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.394374 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.394761 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.394887 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:27.398068 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:27.398073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:27.398076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:27.398107 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:27.400694 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:27.401454 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:27.401459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:27.401462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:27.401583 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:27.401999 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:27.402004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:27.402007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:27.402351 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:27.402501 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:27.403199 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 03:05:27.403450 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":239,"threads_started":1}
I20260807 03:05:27.404829 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:27.386959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42265 (local address 127.5.122.2:48703)
0807 03:05:27.387670 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:27.387708 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:27.389333 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:27.390032 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:27.390040 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:27.390583 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:27.391362 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:27.391386 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.392805 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.392810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:27.394215 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:27.394228 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.394976 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.394983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:27.395201 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:27.395826 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:27.395849 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:27.397848 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:27.400929 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:27.400936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:27.400949 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:27.401340 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:27.401685 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:27.401690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:27.401694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:27.401894 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:27.402482 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:27.402490 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:27.402852 (+ 362us) client_negotiation.cc:769] Sending connection context
0807 03:05:27.403101 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 03:05:27.403611 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":290,"threads_started":1}
I20260807 03:05:27.406386 24392 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42265
I20260807 03:05:27.406730 24392 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:27.407444 24392 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
I20260807 03:05:27.408965 23823 ts_manager.cc:194] Registered new tserver with Master: fd675f6727d54fa8bd54d30fb55fb3bf (127.5.122.2:44285)
I20260807 03:05:27.409979 23823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:48703
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:05:27.451716 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42265
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:27.770550 24405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:27.770823 24405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:27.770957 24405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:27.778389 24405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:27.778509 24405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:27.778657 24405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:05:27.788223 24405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.122.20:34359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:27.789479 24405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:27.790745 24405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:27.803350 24413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:27.803375 24414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:27.803793 24418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:27.804664 24405 server_base.cc:1034] running on GCE node
I20260807 03:05:27.805346 24405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:27.806500 24405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:27.807765 24405 hybrid_clock.cc:648] HybridClock initialized: now 1786071927807678 us; error 73 us; skew 500 ppm
I20260807 03:05:27.808117 24405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:27 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:27.811770 24405 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:05:27.813167 24405 webserver.cc:460] Webserver started at http://127.5.122.3:45785/ using document root <none> and password file <none>
I20260807 03:05:27.813833 24405 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:27.813954 24405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:27.814247 24405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 03:05:27 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:27.830874 24405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "702d237ff1cb479fa1221ee52b1e4e82"
format_stamp: "Formatted at 2026-08-07 03:05:27 on dist-test-slave-fgb3"
server_key: "0c99f420e215ce1e40b7234785046446"
server_key_iv: "900636578759c8825378cb0341ce0612"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:27.831985 24405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "702d237ff1cb479fa1221ee52b1e4e82"
format_stamp: "Formatted at 2026-08-07 03:05:27 on dist-test-slave-fgb3"
server_key: "0c99f420e215ce1e40b7234785046446"
server_key_iv: "900636578759c8825378cb0341ce0612"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:27.838721 24405 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 03:05:27.854439 24427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:27.855855 24405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:05:27.856022 24405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "702d237ff1cb479fa1221ee52b1e4e82"
format_stamp: "Formatted at 2026-08-07 03:05:27 on dist-test-slave-fgb3"
server_key: "0c99f420e215ce1e40b7234785046446"
server_key_iv: "900636578759c8825378cb0341ce0612"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:27.856230 24405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:27.873782 24405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:27.877664 24405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:27.877930 24405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:27.878909 24405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:27.880241 24405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:27.880318 24405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:27.880443 24405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:27.880496 24405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:27.913949 24405 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:39949
I20260807 03:05:27.913981 24599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:39949 every 8 connection(s)
I20260807 03:05:27.915365 24405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:05:27.915881 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 24405
I20260807 03:05:27.916047 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:05:27.916649 5608 external_mini_cluster.cc:1428] Setting key 26b3de0ac83fe4346a9d096daf2e4e6c
Aug 07 03:05:27 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:27.940574 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:27.919342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:34057 (local address 127.5.122.62:42265)
0807 03:05:27.922948 (+ 3606us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:27.922960 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:27.923684 (+ 724us) server_negotiation.cc:407] Connection header received
0807 03:05:27.926079 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:27.926083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:27.926146 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:27.926235 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:27.928035 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.928773 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.931047 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.931691 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.931798 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:27.935016 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:27.935022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:27.935025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:27.935055 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:27.937631 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:27.938335 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:27.938341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:27.938345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:27.938475 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:27.938893 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:27.938899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:27.938902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:27.939211 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:27.939341 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:27.940023 (+ 682us) server_negotiation.cc:299] Negotiation successful
0807 03:05:27.940237 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3326,"thread_start_us":3173,"threads_started":1}
I20260807 03:05:27.941439 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:27.918740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42265 (local address 127.5.122.3:34057)
0807 03:05:27.923508 (+ 4768us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:27.923560 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:27.925820 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:27.926525 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:27.926536 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:27.927115 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:27.927874 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:27.927898 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.928930 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.928935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:27.930918 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:27.930933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.931907 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.931915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:27.932146 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:27.932823 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:27.932844 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:27.934801 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:27.937756 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:27.937765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:27.937780 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:27.938189 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:27.938578 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:27.938583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:27.938587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:27.938772 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:27.939332 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:27.939340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:27.939628 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 03:05:27.939892 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 03:05:27.940350 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4483,"thread_start_us":4239,"threads_started":1}
I20260807 03:05:27.943235 24600 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42265
I20260807 03:05:27.943562 24600 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:27.944361 24600 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
I20260807 03:05:27.945876 23823 ts_manager.cc:194] Registered new tserver with Master: 702d237ff1cb479fa1221ee52b1e4e82 (127.5.122.3:39949)
I20260807 03:05:27.946666 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:05:27.946905 23823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:34057
I20260807 03:05:27.949122 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 23950
Aug 07 03:05:27 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:05:27.972388 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:27.950428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.5.122.1:35427)
0807 03:05:27.957367 (+ 6939us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:27.957374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:27.957398 (+ 24us) server_negotiation.cc:407] Connection header received
0807 03:05:27.957451 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:27.957457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:27.957622 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:27.957770 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:27.959287 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.960407 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.961873 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:27.962424 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:27.962530 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:27.967067 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:27.967078 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:27.967090 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:27.967124 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:27.969720 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:27.970519 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:27.970525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:27.970528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:27.970671 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:27.971051 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:27.971054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:27.971056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:27.971340 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:27.971561 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:27.971874 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 03:05:27.972062 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6785,"thread_start_us":6639,"threads_started":1}
I20260807 03:05:27.973789 24164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:28.060029 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 23950
I20260807 03:05:28.080173 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 24181
Aug 07 03:05:28 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:05:28.097117 24610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:28.081117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.5.122.2:44285)
0807 03:05:28.081654 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:28.081661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:28.081688 (+ 27us) server_negotiation.cc:407] Connection header received
0807 03:05:28.081755 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:28.081762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:28.081994 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:28.082125 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:28.084101 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:28.085213 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:28.086643 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:28.087184 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:28.087286 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:28.091939 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:28.091951 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:28.091965 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:28.092000 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:28.094759 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:28.095409 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:28.095415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:28.095418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:28.095544 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:28.095798 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:28.095801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:28.095803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:28.096121 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:28.096368 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:28.096577 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 03:05:28.096770 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":267,"threads_started":1}
I20260807 03:05:28.098304 24385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:28.180228 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 24181
I20260807 03:05:28.197811 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 24405
Aug 07 03:05:28 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:05:28.214640 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:28.198861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.5.122.3:39949)
0807 03:05:28.199428 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:28.199435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:28.199462 (+ 27us) server_negotiation.cc:407] Connection header received
0807 03:05:28.199521 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:28.199526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:28.199726 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:28.199888 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:28.201199 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:28.202254 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:28.203648 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:28.204526 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:28.204658 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:28.209298 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:28.209311 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:28.209327 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:28.209366 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:28.212022 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:28.212646 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:28.212653 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:28.212655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:28.212791 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:28.213106 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:28.213109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:28.213111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:28.213434 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:28.213690 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:28.213904 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 03:05:28.214104 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":268,"threads_started":1}
I20260807 03:05:28.215899 24593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:28.296583 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 24405
I20260807 03:05:28.315419 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 23769
I20260807 03:05:28.316632 23869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:28.420764 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 23769
I20260807 03:05:28.450742 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42265
--webserver_interface=127.5.122.62
--webserver_port=37077
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:42265
--ranger_config_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:28.869513 24618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:28.869760 24618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:28.869849 24618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:28.876765 24618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:28.876859 24618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:28.876910 24618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:28.876964 24618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:28.877009 24618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:28.886408 24618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:42265
--ranger_config_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=37077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:28.887454 24618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:28.888622 24618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:28.899127 24627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:28.899178 24634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:28.899209 24629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:28.900130 24618 server_base.cc:1034] running on GCE node
I20260807 03:05:28.900807 24618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:28.902024 24618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:28.903260 24618 hybrid_clock.cc:648] HybridClock initialized: now 1786071928903191 us; error 29 us; skew 500 ppm
I20260807 03:05:28.903626 24618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:28 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071928, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:28.906708 24618 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:05:28.908118 24618 webserver.cc:460] Webserver started at http://127.5.122.62:37077/ using document root <none> and password file <none>
I20260807 03:05:28.908743 24618 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:28.908877 24618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 03:05:28 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071928, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:28.953810 24618 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 03:05:28.956859 24643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:28.958258 24618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:05:28.958451 24618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc"
format_stamp: "Formatted at 2026-08-07 03:05:24 on dist-test-slave-fgb3"
server_key: "f8842b6f086b2e433f953a25a7df66e8"
server_key_iv: "2f8db1a2bea3f98ff832f64cad09706b"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:28.959352 24618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:28.986025 24618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:28.989898 24618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:28.990168 24618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:29.007129 24618 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:42265
I20260807 03:05:29.007133 24723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:42265 every 8 connection(s)
I20260807 03:05:29.008569 24618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:05:29.015129 24728 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:05:29.016196 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 24618
I20260807 03:05:29.017028 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:35427
--local_ip_for_outbound_sockets=127.5.122.1
--tserver_master_addrs=127.5.122.62:42265
--webserver_port=37451
--webserver_interface=127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:05:29.018306 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Bootstrap starting.
I20260807 03:05:29.036953 24728 log.cc:824] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:29.045527 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:05:29.046005 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Bootstrap complete.
I20260807 03:05:29.054513 24728 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:29.055336 24728 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8af6fbfd1d54bdda2efd19f78ca94bc, State: Initialized, Role: FOLLOWER
I20260807 03:05:29.055922 24728 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:29.056149 24728 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:05:29.056253 24728 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:05:29.056409 24728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 1 FOLLOWER]: Advancing to term 2
I20260807 03:05:29.060143 24728 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:29.060585 24728 leader_election.cc:302] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8af6fbfd1d54bdda2efd19f78ca94bc; no voters:
I20260807 03:05:29.061146 24728 leader_election.cc:288] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 03:05:29.061286 24733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 2 FOLLOWER]: Leader election won for term 2
I20260807 03:05:29.062381 24733 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [term 2 LEADER]: Becoming Leader. State: Replica: c8af6fbfd1d54bdda2efd19f78ca94bc, State: Running, Role: LEADER
I20260807 03:05:29.063040 24733 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } }
I20260807 03:05:29.063215 24728 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:29.065466 24737 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8af6fbfd1d54bdda2efd19f78ca94bc. Latest consensus state: current_term: 2 leader_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } } }
I20260807 03:05:29.065840 24737 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:29.066394 24735 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8af6fbfd1d54bdda2efd19f78ca94bc" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42265 } } }
I20260807 03:05:29.066673 24735 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:29.075366 24744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:05:29.081152 24744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:05:29.082526 24744 catalog_manager.cc:1259] Loaded cluster ID: b7705f764ed841ffaf82875823aee88e
I20260807 03:05:29.082669 24744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:05:29.086539 24744 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:05:29.087940 24744 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Loaded TSK: 0
I20260807 03:05:29.089509 24744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:05:29.090996 24728 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 03:05:29.457401 24731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:29.457938 24731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:29.458174 24731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:29.469486 24731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:29.469714 24731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:29.469982 24731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:05:29.485630 24731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:35427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=37451
--tserver_master_addrs=127.5.122.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:29.487874 24731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:29.489750 24731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:29.509298 24790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:29.512315 24791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:29.519019 24793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:29.520124 24731 server_base.cc:1034] running on GCE node
I20260807 03:05:29.520852 24731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:29.522056 24731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:29.523314 24731 hybrid_clock.cc:648] HybridClock initialized: now 1786071929523241 us; error 62 us; skew 500 ppm
I20260807 03:05:29.523674 24731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:29 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:29.527874 24731 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:05:29.529759 24731 webserver.cc:460] Webserver started at http://127.5.122.1:37451/ using document root <none> and password file <none>
I20260807 03:05:29.530750 24731 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:29.530926 24731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 03:05:29 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:29.561704 24731 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.007s sys 0.000s
I20260807 03:05:29.565066 24804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:29.566563 24731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:05:29.566736 24731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "aaa1729f451144d78ce59397b6875b37"
format_stamp: "Formatted at 2026-08-07 03:05:25 on dist-test-slave-fgb3"
server_key: "f724db8f791426a9e0de3ddfc95bd4be"
server_key_iv: "aa0b4bfc5acb1b615d8dc226b1334529"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:29.567960 24731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:29.585464 24731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:29.589919 24731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:29.590185 24731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:29.594712 24731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:29.596295 24731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:29.596418 24731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:29.596546 24731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:29.596639 24731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:29.671949 24731 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:35427
I20260807 03:05:29.672008 24987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:35427 every 8 connection(s)
I20260807 03:05:29.673686 24731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:05:29.683686 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 24731
I20260807 03:05:29.684542 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:44285
--local_ip_for_outbound_sockets=127.5.122.2
--tserver_master_addrs=127.5.122.62:42265
--webserver_port=44253
--webserver_interface=127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:05:29 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:29.728642 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:29.677278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42265 (local address 127.5.122.1:59693)
0807 03:05:29.699525 (+ 22247us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:29.699578 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:29.704651 (+ 5073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:29.707089 (+ 2438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:29.707100 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:29.707695 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:29.708542 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:29.708581 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:29.710962 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:29.710968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:29.713016 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:29.713048 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:29.714495 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:29.714502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:29.714782 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:29.715703 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:29.715771 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:29.718822 (+ 3051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:29.723368 (+ 4546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:29.723377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:29.723392 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:29.724001 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:29.724575 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:29.724581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:29.724584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:29.724851 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:29.726033 (+ 1182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:29.726041 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:29.726444 (+ 403us) client_negotiation.cc:769] Sending connection context
0807 03:05:29.726765 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 03:05:29.727294 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21942,"thread_start_us":21699,"threads_started":1}
I20260807 03:05:29.731212 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:29.678532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:59693 (local address 127.5.122.62:42265)
0807 03:05:29.698856 (+ 20324us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:29.698884 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:29.700538 (+ 1654us) server_negotiation.cc:407] Connection header received
0807 03:05:29.705370 (+ 4832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:29.705391 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:29.705901 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:29.706359 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:29.708825 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:29.710751 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:29.713210 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:29.714210 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:29.714465 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:29.719112 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:29.719123 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:29.719142 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:29.719330 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:29.723191 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:29.724196 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:29.724207 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:29.724215 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:29.724430 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:29.725025 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:29.725031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:29.725035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:29.725738 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:29.725944 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:29.729614 (+ 3670us) server_negotiation.cc:299] Negotiation successful
0807 03:05:29.730020 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20034,"thread_start_us":19802,"threads_started":1}
I20260807 03:05:29.741231 24988 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42265
I20260807 03:05:29.741561 24988 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:29.742431 24988 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
W20260807 03:05:29.745765 24988 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:42265 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 03:05:29.745994 24988 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:29.746256 24988 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
I20260807 03:05:29.747474 24988 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:29.747740 24988 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
W20260807 03:05:29.748857 24988 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 03:05:30.108804 24995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:30.109149 24995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:30.109323 24995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:30.120453 24995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:30.120592 24995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:30.120791 24995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:05:30.135886 24995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:44285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=44253
--tserver_master_addrs=127.5.122.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:30.137588 24995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:30.139343 24995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:30.165580 25014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:30.167922 25013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:30.169723 24995 server_base.cc:1034] running on GCE node
W20260807 03:05:30.171707 25021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:30.172652 24995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:30.190626 24995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:30.194114 24995 hybrid_clock.cc:648] HybridClock initialized: now 1786071930194019 us; error 90 us; skew 500 ppm
I20260807 03:05:30.194615 24995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:30 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:30.199082 24995 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:05:30.201149 24995 webserver.cc:460] Webserver started at http://127.5.122.2:44253/ using document root <none> and password file <none>
I20260807 03:05:30.202098 24995 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:30.202258 24995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 03:05:30 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:30.232259 24995 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.008s sys 0.000s
I20260807 03:05:30.236132 25036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:30.237919 24995 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 03:05:30.238116 24995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "fd675f6727d54fa8bd54d30fb55fb3bf"
format_stamp: "Formatted at 2026-08-07 03:05:26 on dist-test-slave-fgb3"
server_key: "aa43f1b55b71db71b81ceda64dd66853"
server_key_iv: "df8c46c6f01b86394b92d008d9fef3db"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:30.239566 24995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 03:05:30 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:30.278419 24995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:30.283601 24995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:30.283897 24995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:30.285097 24995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:30.286864 24995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:30.286980 24995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:30.287086 24995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:30.287163 24995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:30.355789 24995 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:44285
I20260807 03:05:30.355872 25223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:44285 every 8 connection(s)
I20260807 03:05:30.358480 24995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:05:30.368697 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 24995
I20260807 03:05:30.369517 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.20:34359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:39949
--local_ip_for_outbound_sockets=127.5.122.3
--tserver_master_addrs=127.5.122.62:42265
--webserver_port=45785
--webserver_interface=127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:05:30 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:30.387161 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:30.362591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:45509 (local address 127.5.122.62:42265)
0807 03:05:30.364043 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:30.364050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:30.365511 (+ 1461us) server_negotiation.cc:407] Connection header received
0807 03:05:30.368510 (+ 2999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:30.368517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:30.368602 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:30.368701 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:30.370742 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:30.372001 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:30.374469 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:30.375136 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:30.375231 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:30.380120 (+ 4889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:30.380126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:30.380130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:30.380169 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:30.383548 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:30.384413 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:30.384418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:30.384421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:30.384560 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:30.385120 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:30.385125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:30.385129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:30.385451 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:30.385577 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:30.386547 (+ 970us) server_negotiation.cc:299] Negotiation successful
0807 03:05:30.386788 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1006,"thread_start_us":885,"threads_started":1}
I20260807 03:05:30.388222 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:30.361953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42265 (local address 127.5.122.2:45509)
0807 03:05:30.365183 (+ 3230us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:30.365233 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:30.368103 (+ 2870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:30.369120 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:30.369131 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:30.369749 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:30.370520 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:30.370546 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:30.372198 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:30.372204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:30.374274 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:30.374289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:30.376424 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:30.376431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:30.376680 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:30.377414 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:30.377449 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:30.379892 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:30.383705 (+ 3813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:30.383714 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:30.383730 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:30.384266 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:30.384679 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:30.384684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:30.384688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:30.384978 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:30.385645 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:30.385652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:30.386040 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 03:05:30.386391 (+ 351us) client_negotiation.cc:240] Negotiation successful
0807 03:05:30.386920 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2710,"thread_start_us":2537,"threads_started":1}
I20260807 03:05:30.389904 25224 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42265
I20260807 03:05:30.390237 25224 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:30.391132 25224 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
I20260807 03:05:30.393252 24668 ts_manager.cc:194] Registered new tserver with Master: fd675f6727d54fa8bd54d30fb55fb3bf (127.5.122.2:44285)
I20260807 03:05:30.395529 24668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:45509
Aug 07 03:05:30 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:30.749008 24988 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:30.749440 24988 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
I20260807 03:05:30.750761 24668 ts_manager.cc:194] Registered new tserver with Master: aaa1729f451144d78ce59397b6875b37 (127.5.122.1:35427)
I20260807 03:05:30.751965 24668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:59693
W20260807 03:05:30.806206 25229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:30.806721 25229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:30.806911 25229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:30.817211 25229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:30.817349 25229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:30.817512 25229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:05:30.828176 25229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.20:34359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:39949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=45785
--tserver_master_addrs=127.5.122.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:30.829460 25229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:30.830785 25229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:30.843713 25237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:30.843685 25236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:30.844082 25239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:30.844552 25229 server_base.cc:1034] running on GCE node
I20260807 03:05:30.845111 25229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:30.846215 25229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:30.847532 25229 hybrid_clock.cc:648] HybridClock initialized: now 1786071930847461 us; error 51 us; skew 500 ppm
I20260807 03:05:30.847895 25229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:30 dist-test-slave-fgb3 krb5kdc[23078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:30.851883 25229 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:05:30.855669 25229 webserver.cc:460] Webserver started at http://127.5.122.3:45785/ using document root <none> and password file <none>
I20260807 03:05:30.856511 25229 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:30.856665 25229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 03:05:30 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:30.892532 25229 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.002s
I20260807 03:05:30.895885 25252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:30.897548 25229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:05:30.897758 25229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "702d237ff1cb479fa1221ee52b1e4e82"
format_stamp: "Formatted at 2026-08-07 03:05:27 on dist-test-slave-fgb3"
server_key: "0c99f420e215ce1e40b7234785046446"
server_key_iv: "900636578759c8825378cb0341ce0612"
server_key_version: "kuduclusterkey@0"
I20260807 03:05:30.899118 25229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:30.926618 25229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:30.931511 25229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:30.931792 25229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:30.932971 25229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:30.934612 25229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:30.934725 25229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:30.934834 25229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:30.934904 25229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 03:05:31 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for HTTP/127.5.122.20@KRBTEST.COM
I20260807 03:05:31.044301 25426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:39949 every 8 connection(s)
I20260807 03:05:31.044654 25229 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:39949
I20260807 03:05:31.046494 25229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:05:31.053524 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 25229
Aug 07 03:05:31 dist-test-slave-fgb3 krb5kdc[23078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071930, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:31.117941 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.054518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.5.122.62:42265)
0807 03:05:31.080647 (+ 26129us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.080654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.085508 (+ 4854us) server_negotiation.cc:407] Connection header received
0807 03:05:31.086614 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.086619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.086702 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.086774 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:31.090958 (+ 4184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.092093 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.097810 (+ 5717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.098465 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.098559 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.106839 (+ 8280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:31.106846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:31.106851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:31.106887 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:31.110060 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.114939 (+ 4879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.114944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.114947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.115095 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.115511 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.115515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.115518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.115859 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:31.116086 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.117257 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.117470 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25994,"thread_start_us":26146,"threads_started":1}
I20260807 03:05:31.123183 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.054556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:58761 (local address 127.5.122.62:42265)
0807 03:05:31.084478 (+ 29922us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.084483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.084503 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:05:31.084566 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.084572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.084669 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.084770 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:31.094191 (+ 9421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.095396 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.099680 (+ 4284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.100348 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.100435 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.111013 (+ 10578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:31.111021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:31.111025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:31.111061 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:31.114041 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.116531 (+ 2490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.116536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.116539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.116655 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.119733 (+ 3078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.119737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.119740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.120041 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:31.121499 (+ 1458us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.122586 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.122802 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29796,"thread_start_us":31131,"threads_started":1}
I20260807 03:05:31.125669 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.053938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42265 (local address 127.5.122.3:58761)
0807 03:05:31.081276 (+ 27338us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:31.081329 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:31.083643 (+ 2314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:31.089220 (+ 5577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:31.089233 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:31.089857 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:31.090713 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.090741 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.095605 (+ 4864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.095610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.097640 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.097656 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.100773 (+ 3117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.100780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.101090 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.103677 (+ 2587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:31.103708 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:31.106576 (+ 2868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:31.114207 (+ 7631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:31.114216 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:31.114235 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:31.114790 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:31.116777 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:31.116781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:31.116784 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:31.117044 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:31.121610 (+ 4566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:31.121618 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:31.122049 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 03:05:31.123667 (+ 1618us) client_negotiation.cc:240] Negotiation successful
0807 03:05:31.124265 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27008,"thread_start_us":31480,"threads_started":1}
I20260807 03:05:31.127892 25427 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42265
I20260807 03:05:31.128196 25427 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:31.129006 25427 heartbeater.cc:511] Master 127.5.122.62:42265 requested a full tablet report, sending...
I20260807 03:05:31.130723 24668 ts_manager.cc:194] Registered new tserver with Master: 702d237ff1cb479fa1221ee52b1e4e82 (127.5.122.3:39949)
I20260807 03:05:31.131882 24668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:58761
I20260807 03:05:31.136476 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:05:31.184100 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.166911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.5.122.62:42265)
0807 03:05:31.167087 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.167093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.167846 (+ 753us) server_negotiation.cc:407] Connection header received
0807 03:05:31.168034 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.168040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.168123 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.168225 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:31.170338 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.171423 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.173425 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.174005 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.174104 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.177638 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:31.177645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:31.177649 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:31.177689 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:31.181013 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.181839 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.181844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.181848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.181980 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.182423 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.182428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.182432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.182766 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:31.182977 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.183524 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.183754 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:05:31.190918 24668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:05:31.201749 24668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:05:31.248060 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.218626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:44285 (local address 127.0.0.1:45044)
0807 03:05:31.238668 (+ 20042us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:31.238687 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:31.238805 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:31.239853 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:31.239862 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:31.239894 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:31.240293 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.240305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.242446 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.242452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.244668 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.244686 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.247107 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.247112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.247289 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.247346 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 03:05:31.247478 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 03:05:31.247685 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19937,"thread_start_us":16366,"threads_started":1}
I20260807 03:05:31.248908 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.218872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45044 (local address 127.5.122.2:44285)
0807 03:05:31.234824 (+ 15952us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.234835 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.238933 (+ 4098us) server_negotiation.cc:407] Connection header received
0807 03:05:31.239059 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.239067 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.239364 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.239555 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:31.240513 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.242222 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.244900 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.246680 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.246787 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.246967 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.248445 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.248608 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15300,"thread_start_us":15542,"threads_started":1}
I20260807 03:05:31.252485 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.218434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:35427 (local address 127.0.0.1:52148)
0807 03:05:31.235416 (+ 16982us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:31.235463 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:31.235636 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:31.243856 (+ 8220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:31.243861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:31.243886 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:31.245242 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.245254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.247328 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.247335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.249495 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.249511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.251757 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.251764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.251921 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.251953 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:05:31.252036 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 03:05:31.252158 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16687,"thread_start_us":20746,"threads_started":1}
I20260807 03:05:31.253386 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.219039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.5.122.1:35427)
0807 03:05:31.242861 (+ 23822us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.242869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.242897 (+ 28us) server_negotiation.cc:407] Connection header received
0807 03:05:31.242987 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.242995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.243234 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.243424 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:31.245417 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.247127 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.249731 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.251244 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.251374 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.251599 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.252895 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.253055 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23626,"thread_start_us":24537,"threads_started":1}
I20260807 03:05:31.256458 24885 tablet_service.cc:1467] Processing CreateTablet for tablet d021d33ef2fa4b5980254360f4e3013d (DEFAULT_TABLE table=test-table [id=1c99cd67722c4813b5502f5df957f1a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:31.256537 25110 tablet_service.cc:1467] Processing CreateTablet for tablet d021d33ef2fa4b5980254360f4e3013d (DEFAULT_TABLE table=test-table [id=1c99cd67722c4813b5502f5df957f1a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:31.257717 24885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d021d33ef2fa4b5980254360f4e3013d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:31.257755 25110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d021d33ef2fa4b5980254360f4e3013d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:31.274621 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.245398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.5.122.3:39949)
0807 03:05:31.263935 (+ 18537us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.263945 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.263983 (+ 38us) server_negotiation.cc:407] Connection header received
0807 03:05:31.264082 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.264091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.264349 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.264544 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:31.267363 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.268963 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.271327 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.273054 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.273169 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.273420 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.274009 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.274198 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18310,"thread_start_us":17978,"threads_started":1}
I20260807 03:05:31.275645 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.251279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39949 (local address 127.0.0.1:44702)
0807 03:05:31.252050 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:31.252067 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:31.252150 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:31.266893 (+ 14743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:31.266898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:31.266931 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:31.267133 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.267146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.269152 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.269158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.271064 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.271092 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.273575 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.273581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.273797 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.273834 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 03:05:31.275132 (+ 1298us) client_negotiation.cc:240] Negotiation successful
0807 03:05:31.275315 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670}
I20260807 03:05:31.277448 25459 tablet_bootstrap.cc:492] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Bootstrap starting.
I20260807 03:05:31.280122 25321 tablet_service.cc:1467] Processing CreateTablet for tablet d021d33ef2fa4b5980254360f4e3013d (DEFAULT_TABLE table=test-table [id=1c99cd67722c4813b5502f5df957f1a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:31.281522 25458 tablet_bootstrap.cc:492] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Bootstrap starting.
I20260807 03:05:31.281656 25321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d021d33ef2fa4b5980254360f4e3013d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:31.285156 25459 tablet_bootstrap.cc:654] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:31.287376 25458 tablet_bootstrap.cc:654] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:31.288682 25459 log.cc:824] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:31.290402 25458 log.cc:824] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:31.296960 25458 tablet_bootstrap.cc:492] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: No bootstrap required, opened a new log
I20260807 03:05:31.297362 25458 ts_tablet_manager.cc:1397] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.003s
I20260807 03:05:31.301427 25459 tablet_bootstrap.cc:492] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: No bootstrap required, opened a new log
I20260807 03:05:31.301761 25459 ts_tablet_manager.cc:1397] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.000s
I20260807 03:05:31.312003 25464 tablet_bootstrap.cc:492] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Bootstrap starting.
I20260807 03:05:31.310882 25458 raft_consensus.cc:348] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.313225 25458 raft_consensus.cc:374] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:31.313349 25458 raft_consensus.cc:729] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa1729f451144d78ce59397b6875b37, State: Initialized, Role: FOLLOWER
I20260807 03:05:31.314102 25458 consensus_queue.cc:260] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.315878 25464 tablet_bootstrap.cc:654] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:31.317252 25464 log.cc:824] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:31.320781 25459 raft_consensus.cc:348] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.321326 25459 raft_consensus.cc:374] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:31.321465 25459 raft_consensus.cc:729] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd675f6727d54fa8bd54d30fb55fb3bf, State: Initialized, Role: FOLLOWER
I20260807 03:05:31.322207 25459 consensus_queue.cc:260] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.324756 25458 ts_tablet_manager.cc:1428] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Time spent starting tablet: real 0.026s user 0.018s sys 0.004s
I20260807 03:05:31.325242 24988 heartbeater.cc:503] Master 127.5.122.62:42265 was elected leader, sending a full tablet report...
I20260807 03:05:31.328414 25464 tablet_bootstrap.cc:492] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: No bootstrap required, opened a new log
I20260807 03:05:31.328788 25464 ts_tablet_manager.cc:1397] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260807 03:05:31.329218 25224 heartbeater.cc:503] Master 127.5.122.62:42265 was elected leader, sending a full tablet report...
I20260807 03:05:31.331617 25459 ts_tablet_manager.cc:1428] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Time spent starting tablet: real 0.030s user 0.009s sys 0.008s
I20260807 03:05:31.337253 25464 raft_consensus.cc:348] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.337662 25464 raft_consensus.cc:374] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:31.337750 25464 raft_consensus.cc:729] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 702d237ff1cb479fa1221ee52b1e4e82, State: Initialized, Role: FOLLOWER
I20260807 03:05:31.338351 25464 consensus_queue.cc:260] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.339416 25427 heartbeater.cc:503] Master 127.5.122.62:42265 was elected leader, sending a full tablet report...
I20260807 03:05:31.339977 25464 ts_tablet_manager.cc:1428] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
W20260807 03:05:31.368085 25225 tablet.cc:2369] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:05:31.410724 25473 raft_consensus.cc:482] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:31.411085 25473 raft_consensus.cc:504] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.413133 25473 leader_election.cc:288] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaa1729f451144d78ce59397b6875b37 (127.5.122.1:35427), 702d237ff1cb479fa1221ee52b1e4e82 (127.5.122.3:39949)
I20260807 03:05:31.420635 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.413636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:42119 (local address 127.5.122.1:35427)
0807 03:05:31.414126 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.414133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.415054 (+ 921us) server_negotiation.cc:407] Connection header received
0807 03:05:31.415232 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.415237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.415317 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.415400 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:31.415936 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.416751 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.418476 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.419594 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.419728 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.419845 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.420248 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.420386 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260807 03:05:31.420665 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.413454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:35427 (local address 127.5.122.2:42119)
0807 03:05:31.414947 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:31.414962 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:31.415094 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:31.415419 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:31.415423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:31.415446 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:31.415773 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.415783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.416880 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.416885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.418294 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.418307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.419830 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.419836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.420016 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.420055 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 03:05:31.420201 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 03:05:31.420331 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1388,"thread_start_us":1211,"threads_started":1}
I20260807 03:05:31.420748 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.413865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:56947 (local address 127.5.122.3:39949)
0807 03:05:31.414293 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.414299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.414338 (+ 39us) server_negotiation.cc:407] Connection header received
0807 03:05:31.414495 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.414501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.414576 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.414632 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:31.415366 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.416269 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.418711 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.419861 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.419992 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.420089 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.420404 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.420533 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290}
I20260807 03:05:31.421325 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.413357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39949 (local address 127.5.122.2:56947)
0807 03:05:31.414136 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:31.414157 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:31.414285 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:31.414853 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:31.414857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:31.414884 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:31.415219 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.415230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.416410 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.416415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.418555 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:31.418571 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.420058 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.420064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:31.420228 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.420260 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:05:31.420920 (+ 660us) client_negotiation.cc:240] Negotiation successful
0807 03:05:31.421050 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":984,"threads_started":1}
I20260807 03:05:31.421953 24914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d021d33ef2fa4b5980254360f4e3013d" candidate_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa1729f451144d78ce59397b6875b37" is_pre_election: true
I20260807 03:05:31.422389 25351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d021d33ef2fa4b5980254360f4e3013d" candidate_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702d237ff1cb479fa1221ee52b1e4e82" is_pre_election: true
I20260807 03:05:31.422598 24914 raft_consensus.cc:2454] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd675f6727d54fa8bd54d30fb55fb3bf in term 0.
I20260807 03:05:31.422911 25351 raft_consensus.cc:2454] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd675f6727d54fa8bd54d30fb55fb3bf in term 0.
I20260807 03:05:31.423329 25041 leader_election.cc:302] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa1729f451144d78ce59397b6875b37, fd675f6727d54fa8bd54d30fb55fb3bf; no voters:
I20260807 03:05:31.423748 25473 raft_consensus.cc:2781] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:05:31.423908 25473 raft_consensus.cc:482] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:05:31.424000 25473 raft_consensus.cc:3037] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:31.427062 25473 raft_consensus.cc:504] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
I20260807 03:05:31.427711 25473 leader_election.cc:288] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [CANDIDATE]: Term 1 election: Requested vote from peers aaa1729f451144d78ce59397b6875b37 (127.5.122.1:35427), 702d237ff1cb479fa1221ee52b1e4e82 (127.5.122.3:39949)
I20260807 03:05:31.428121 24914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d021d33ef2fa4b5980254360f4e3013d" candidate_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa1729f451144d78ce59397b6875b37"
I20260807 03:05:31.428151 25351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d021d33ef2fa4b5980254360f4e3013d" candidate_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702d237ff1cb479fa1221ee52b1e4e82"
I20260807 03:05:31.428309 24914 raft_consensus.cc:3037] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:31.428354 25351 raft_consensus.cc:3037] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:31.430435 24914 raft_consensus.cc:2454] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd675f6727d54fa8bd54d30fb55fb3bf in term 1.
I20260807 03:05:31.430805 25041 leader_election.cc:302] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa1729f451144d78ce59397b6875b37, fd675f6727d54fa8bd54d30fb55fb3bf; no voters:
I20260807 03:05:31.431046 25473 raft_consensus.cc:2781] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:05:31.431430 25351 raft_consensus.cc:2454] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd675f6727d54fa8bd54d30fb55fb3bf in term 1.
I20260807 03:05:31.431515 25473 raft_consensus.cc:686] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 1 LEADER]: Becoming Leader. State: Replica: fd675f6727d54fa8bd54d30fb55fb3bf, State: Running, Role: LEADER
I20260807 03:05:31.432233 25473 consensus_queue.cc:237] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } }
W20260807 03:05:31.432363 24989 tablet.cc:2369] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:05:31.436921 24666 catalog_manager.cc:5495] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf reported cstate change: term changed from 0 to 1, leader changed from <none> to fd675f6727d54fa8bd54d30fb55fb3bf (127.5.122.2). New cstate: current_term: 1 leader_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd675f6727d54fa8bd54d30fb55fb3bf" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 } health_report { overall_health: UNKNOWN } } }
I20260807 03:05:31.469869 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.463753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.5.122.2:44285)
0807 03:05:31.463928 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.463933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.463957 (+ 24us) server_negotiation.cc:407] Connection header received
0807 03:05:31.464274 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.464280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.464398 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.464549 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:05:31.464886 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.466018 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.467390 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.468055 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.468179 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.468605 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:05:31.469277 (+ 672us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:05:31.469435 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.469485 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.469566 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 03:05:31.477944 25351 raft_consensus.cc:1261] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 1 FOLLOWER]: Refusing update from remote peer fd675f6727d54fa8bd54d30fb55fb3bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:31.478022 24914 raft_consensus.cc:1261] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 1 FOLLOWER]: Refusing update from remote peer fd675f6727d54fa8bd54d30fb55fb3bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:31.478967 25478 consensus_queue.cc:1035] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "702d237ff1cb479fa1221ee52b1e4e82" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:05:31.479283 25473 consensus_queue.cc:1035] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaa1729f451144d78ce59397b6875b37" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:05:31.490767 25483 mvcc.cc:205] Tried to move back new op lower bound from 7315750631324041216 to 7315750631150301184. Current Snapshot: MvccSnapshot[applied={T|T < 7315750631324041216}]
I20260807 03:05:31.507292 24668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:05:31.507562 24668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:05:31.511693 24668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8af6fbfd1d54bdda2efd19f78ca94bc: Sending DeleteTablet for 3 replicas of tablet d021d33ef2fa4b5980254360f4e3013d
I20260807 03:05:31.512756 24885 tablet_service.cc:1514] Processing DeleteTablet for tablet d021d33ef2fa4b5980254360f4e3013d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:31 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:52148
I20260807 03:05:31.513139 25110 tablet_service.cc:1514] Processing DeleteTablet for tablet d021d33ef2fa4b5980254360f4e3013d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:31 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:45044
I20260807 03:05:31.513897 25500 tablet_replica.cc:330] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: stopping tablet replica
I20260807 03:05:31.514030 25321 tablet_service.cc:1514] Processing DeleteTablet for tablet d021d33ef2fa4b5980254360f4e3013d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:31 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:44702
I20260807 03:05:31.514334 25502 tablet_replica.cc:330] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: stopping tablet replica
I20260807 03:05:31.514367 25500 raft_consensus.cc:2229] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:05:31.514685 25502 raft_consensus.cc:2229] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:05:31.514822 25500 raft_consensus.cc:2258] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:31.515273 25502 raft_consensus.cc:2258] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:31.516824 25502 ts_tablet_manager.cc:1905] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:31.519212 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 24731
I20260807 03:05:31.521474 25504 tablet_replica.cc:330] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: stopping tablet replica
I20260807 03:05:31.522032 25504 raft_consensus.cc:2229] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:05:31.522554 25504 raft_consensus.cc:2258] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:31.524782 25500 ts_tablet_manager.cc:1905] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:31.526240 25502 ts_tablet_manager.cc:1918] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:31.526886 25504 ts_tablet_manager.cc:1905] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:31.527493 25502 log.cc:1196] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/d021d33ef2fa4b5980254360f4e3013d
I20260807 03:05:31.528142 25502 ts_tablet_manager.cc:1939] T d021d33ef2fa4b5980254360f4e3013d P fd675f6727d54fa8bd54d30fb55fb3bf: Deleting consensus metadata
I20260807 03:05:31.530023 24644 catalog_manager.cc:4841] TS fd675f6727d54fa8bd54d30fb55fb3bf (127.5.122.2:44285): tablet d021d33ef2fa4b5980254360f4e3013d (table test-table [id=1c99cd67722c4813b5502f5df957f1a1]) successfully deleted
I20260807 03:05:31.531685 25500 ts_tablet_manager.cc:1918] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:31.531939 25500 log.cc:1196] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/d021d33ef2fa4b5980254360f4e3013d
I20260807 03:05:31.532591 25500 ts_tablet_manager.cc:1939] T d021d33ef2fa4b5980254360f4e3013d P aaa1729f451144d78ce59397b6875b37: Deleting consensus metadata
I20260807 03:05:31.534584 24648 catalog_manager.cc:4841] TS aaa1729f451144d78ce59397b6875b37 (127.5.122.1:35427): tablet d021d33ef2fa4b5980254360f4e3013d (table test-table [id=1c99cd67722c4813b5502f5df957f1a1]) successfully deleted
I20260807 03:05:31.537559 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.521554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.5.122.1:35427)
0807 03:05:31.521706 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.521711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.521728 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:05:31.521877 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.521883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.521956 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.522068 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:31.523732 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.524722 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.526489 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.527140 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.527268 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.530500 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:31.530515 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:31.530530 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:31.530572 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:31.534556 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.535243 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.535253 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.535256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.535416 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.535876 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.535882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.535885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.536327 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:31.536622 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.536970 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.537182 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:05:31.537765 25504 ts_tablet_manager.cc:1918] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:31.537927 25504 log.cc:1196] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/d021d33ef2fa4b5980254360f4e3013d
I20260807 03:05:31.538517 25504 ts_tablet_manager.cc:1939] T d021d33ef2fa4b5980254360f4e3013d P 702d237ff1cb479fa1221ee52b1e4e82: Deleting consensus metadata
I20260807 03:05:31.538568 24980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:31.540454 24644 catalog_manager.cc:4841] TS 702d237ff1cb479fa1221ee52b1e4e82 (127.5.122.3:39949): tablet d021d33ef2fa4b5980254360f4e3013d (table test-table [id=1c99cd67722c4813b5502f5df957f1a1]) successfully deleted
I20260807 03:05:31.648428 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 24731
I20260807 03:05:31.669665 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 24995
I20260807 03:05:31.689572 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.670680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.5.122.2:44285)
0807 03:05:31.670837 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.670842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.670957 (+ 115us) server_negotiation.cc:407] Connection header received
0807 03:05:31.671169 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.671174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.671259 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.671356 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:31.672764 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.673608 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.675071 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.675597 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.675746 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.682859 (+ 7113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:31.682874 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:31.682890 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:31.682930 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:31.686746 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.687424 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.687431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.687434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.687590 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.687979 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.687983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.687985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.688364 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:31.688699 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.688995 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.689215 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 03:05:31.690343 25215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:31.800704 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 24995
I20260807 03:05:31.821441 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 25229
I20260807 03:05:31.833830 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:31.822635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.5.122.3:39949)
0807 03:05:31.822822 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:31.822827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:31.822846 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:05:31.822908 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:31.822911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:31.822975 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:31.823054 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:31.824223 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.824945 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.826120 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:31.826607 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:31.826701 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:31.828897 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:31.828910 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:31.828925 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:31.828958 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:31.831736 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.832291 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.832298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.832300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.832423 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:31.832694 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:31.832697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:31.832699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:31.833043 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:31.833262 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:31.833435 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 03:05:31.833587 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:05:31.834705 25419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:31.919548 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 25229
I20260807 03:05:31.939429 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 24618
I20260807 03:05:31.940541 24713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:32.049131 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 24618
I20260807 03:05:32.072804 5608 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 03:05:32.416577 5608 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 03:05:32.416777 5608 mini_ranger.cc:66] Stopping Ranger...
I20260807 03:05:32.760715 5608 mini_ranger.cc:68] Stopped Ranger
2026-08-07 03:05:32.760 UTC [23124] LOG: received smart shutdown request
2026-08-07 03:05:32.763 UTC [23124] LOG: background worker "logical replication launcher" (PID 23138) exited with exit code 1
2026-08-07 03:05:32.764 UTC [23131] LOG: shutting down
2026-08-07 03:05:32.892 UTC [23124] LOG: database system is shut down
2026-08-07T03:05:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50771 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:05:33 dist-test-slave-fgb3 krb5kdc[25522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:05:33 dist-test-slave-fgb3 krb5kdc[25522](info): set up 2 sockets
Aug 07 03:05:33 dist-test-slave-fgb3 krb5kdc[25522](info): commencing operation
krb5kdc: starting...
W20260807 03:05:34.202394 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:05:34 dist-test-slave-fgb3 krb5kdc[25522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:05:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:05:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:05:34.455415 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39451
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39451,127.5.122.61:36031,127.5.122.60:45715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:34.774564 25540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:34.774816 25540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:34.774910 25540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:34.781765 25540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:34.781857 25540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:34.781910 25540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:34.781965 25540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:34.782009 25540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:34.791114 25540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39451,127.5.122.61:36031,127.5.122.60:45715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:34.792209 25540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:34.793334 25540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:34.804180 25548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:34.804244 25547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:34.804266 25551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:34.805058 25540 server_base.cc:1034] running on GCE node
I20260807 03:05:34.805742 25540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:34.807078 25540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:34.808281 25540 hybrid_clock.cc:648] HybridClock initialized: now 1786071934808245 us; error 27 us; skew 500 ppm
I20260807 03:05:34.808593 25540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:34 dist-test-slave-fgb3 krb5kdc[25522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:34.811542 25540 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:05:34.812911 25540 webserver.cc:460] Webserver started at http://127.5.122.62:44607/ using document root <none> and password file <none>
I20260807 03:05:34.813525 25540 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:34.813642 25540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:34.813925 25540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:34.816763 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "876f3a8cad52454fa4f292aa96378458"
format_stamp: "Formatted at 2026-08-07 03:05:34 on dist-test-slave-fgb3"
server_key: "c474ef7a1eb35469ffddf63105761370"
server_key_iv: "a9a39e60441cdc18fecac2d60f72ee3a"
server_key_version: "encryptionkey@0"
I20260807 03:05:34.817538 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "876f3a8cad52454fa4f292aa96378458"
format_stamp: "Formatted at 2026-08-07 03:05:34 on dist-test-slave-fgb3"
server_key: "c474ef7a1eb35469ffddf63105761370"
server_key_iv: "a9a39e60441cdc18fecac2d60f72ee3a"
server_key_version: "encryptionkey@0"
I20260807 03:05:34.822286 25540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 03:05:34.825541 25563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:34.826751 25540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:05:34.826926 25540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "876f3a8cad52454fa4f292aa96378458"
format_stamp: "Formatted at 2026-08-07 03:05:34 on dist-test-slave-fgb3"
server_key: "c474ef7a1eb35469ffddf63105761370"
server_key_iv: "a9a39e60441cdc18fecac2d60f72ee3a"
server_key_version: "encryptionkey@0"
I20260807 03:05:34.827097 25540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:34.845772 25540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:34.849670 25540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:34.849920 25540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:34.866608 25540 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:39451
I20260807 03:05:34.866647 25643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:39451 every 8 connection(s)
I20260807 03:05:34.867889 25540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:05:34.871462 25646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:34.876109 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 25540
I20260807 03:05:34.876366 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:05:34.877177 5608 external_mini_cluster.cc:1428] Setting key ee5ec55034997e43d5f7dc1b2f5c395a
I20260807 03:05:34.878607 25646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:05:34 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:34.901295 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:34.881885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39451 (local address 127.0.0.1:60110)
0807 03:05:34.882697 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:34.882745 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:34.884995 (+ 2250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:34.885778 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:34.885785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:34.886509 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:34.886950 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:34.886973 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:34.888509 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:34.888514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:34.889813 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:34.889826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:34.890981 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:34.890987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:34.891156 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:34.891743 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:34.891764 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:34.893896 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:34.897606 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:34.897616 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:34.897624 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:34.898023 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:34.898456 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:34.898459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:34.898462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:34.898613 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:34.899464 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:34.899469 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:34.899648 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 03:05:34.899841 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 03:05:34.900202 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":6,"thread_start_us":299,"threads_started":1}
I20260807 03:05:34.901322 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:34.882948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.5.122.62:39451)
0807 03:05:34.883534 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:34.883540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:34.883581 (+ 41us) server_negotiation.cc:407] Connection header received
0807 03:05:34.885329 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:34.885335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:34.885550 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:34.885698 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:34.887139 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:34.888338 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:34.889945 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:34.890766 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:34.890927 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:34.894186 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:34.894197 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:34.894213 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:34.894252 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:34.897489 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:34.898145 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:34.898153 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:34.898157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:34.898353 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:34.898722 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:34.898726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:34.898729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:34.899324 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:34.899523 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:34.900006 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 03:05:34.900358 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":441,"thread_start_us":292,"threads_started":1}
I20260807 03:05:34.904384 25646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.122.61@KRBTEST.COM; defaulting to no policy
I20260807 03:05:34.905421 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:34.905079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37008)
0807 03:05:34.905220 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:34.905320 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 03:05:34.905666 25569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
I20260807 03:05:34.907106 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:34.906750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37018)
0807 03:05:34.906906 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:34.906997 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 03:05:34.907267 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:36031: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab.
Entry for principal kudu/127.5.122.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab.
I20260807 03:05:34.934474 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:36031
--webserver_interface=127.5.122.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab
--principal=kudu/127.5.122.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39451,127.5.122.61:36031,127.5.122.60:45715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:05:34.956687 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } attempt: 1
I20260807 03:05:34.957921 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:34.957432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37032)
0807 03:05:34.957650 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:34.957770 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 03:05:34.958907 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:34.958596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37036)
0807 03:05:34.958711 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:34.958775 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 03:05:34.959071 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:36031: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
I20260807 03:05:35.027283 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } attempt: 2
I20260807 03:05:35.028460 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.028002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37048)
0807 03:05:35.028228 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.028320 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 03:05:35.029409 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.029082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37056)
0807 03:05:35.029193 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.029273 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 03:05:35.029596 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:36031: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
I20260807 03:05:35.120821 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } attempt: 3
I20260807 03:05:35.122044 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.121598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37064)
0807 03:05:35.121794 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.121911 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 03:05:35.123044 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.122683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37078)
0807 03:05:35.122824 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.122904 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 03:05:35.123221 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:36031: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
W20260807 03:05:35.255659 25655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:35.255967 25655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:35.256106 25655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:35.263379 25655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:35.263502 25655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:35.263583 25655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:35.263664 25655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:35.263725 25655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:35.266438 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } attempt: 4
I20260807 03:05:35.267513 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.267003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37092)
0807 03:05:35.267292 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.267381 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260807 03:05:35.268467 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.268114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37100)
0807 03:05:35.268254 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.268343 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 03:05:35.268699 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:36031: Network error: Client connection negotiation failed: client connection to 127.5.122.61:36031: connect: Connection refused (error 111)
I20260807 03:05:35.273914 25655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39451,127.5.122.61:36031,127.5.122.60:45715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab
--principal=kudu/127.5.122.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:36031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:35.275281 25655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:35.276563 25655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:35.288048 25664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:35.288009 25667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:35.288069 25662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:35.289005 25655 server_base.cc:1034] running on GCE node
I20260807 03:05:35.289783 25655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:35.291239 25655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:35.292435 25655 hybrid_clock.cc:648] HybridClock initialized: now 1786071935292390 us; error 34 us; skew 500 ppm
I20260807 03:05:35.292768 25655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:35.295848 25655 init.cc:377] Logged in from keytab as kudu/127.5.122.61@KRBTEST.COM (short username kudu)
I20260807 03:05:35.297261 25655 webserver.cc:460] Webserver started at http://127.5.122.61:37775/ using document root <none> and password file <none>
I20260807 03:05:35.297940 25655 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:35.298070 25655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:35.298410 25655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:35.301383 25655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data/instance:
uuid: "91d58966d3494b20896be6b88da04f37"
format_stamp: "Formatted at 2026-08-07 03:05:35 on dist-test-slave-fgb3"
server_key: "58074b8ff18262aa1e656e7a99e11d37"
server_key_iv: "db3258022c19fd4595c0ed3dea0c06fb"
server_key_version: "encryptionkey@0"
I20260807 03:05:35.302172 25655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/wal/instance:
uuid: "91d58966d3494b20896be6b88da04f37"
format_stamp: "Formatted at 2026-08-07 03:05:35 on dist-test-slave-fgb3"
server_key: "58074b8ff18262aa1e656e7a99e11d37"
server_key_iv: "db3258022c19fd4595c0ed3dea0c06fb"
server_key_version: "encryptionkey@0"
I20260807 03:05:35.307286 25655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 03:05:35.310575 25676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:35.312028 25655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:05:35.312211 25655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/wal
uuid: "91d58966d3494b20896be6b88da04f37"
format_stamp: "Formatted at 2026-08-07 03:05:35 on dist-test-slave-fgb3"
server_key: "58074b8ff18262aa1e656e7a99e11d37"
server_key_iv: "db3258022c19fd4595c0ed3dea0c06fb"
server_key_version: "encryptionkey@0"
I20260807 03:05:35.312393 25655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:35.336440 25655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:35.340317 25655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:35.340583 25655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:35.357682 25655 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.61:36031
I20260807 03:05:35.357712 25755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.61:36031 every 8 connection(s)
I20260807 03:05:35.359115 25655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
I20260807 03:05:35.360409 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 25655
I20260807 03:05:35.360597 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-1/wal/instance
I20260807 03:05:35.361176 5608 external_mini_cluster.cc:1428] Setting key 722d61a5dba84880344f4450b3cb371d
I20260807 03:05:35.363662 25756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:35.372006 25756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.60@KRBTEST.COM" created.
I20260807 03:05:35.392575 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.374647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.5.122.62:39451)
0807 03:05:35.374867 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.374873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.376050 (+ 1177us) server_negotiation.cc:407] Connection header received
0807 03:05:35.377833 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.377839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.377932 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.378088 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.379481 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.380474 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.381927 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.382583 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.382695 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.386692 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.386698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.386703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.386744 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.389560 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.390401 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.390407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.390410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.390530 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.390937 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.390941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.390943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.391215 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.391358 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.392017 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.392240 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 03:05:35.393438 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.375086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39451 (local address 127.0.0.1:60112)
0807 03:05:35.375870 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.375918 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:35.377570 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:35.378242 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:35.378249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:35.378778 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:35.379315 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.379332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.380605 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.380609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.381786 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.381798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.382983 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.382991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.383263 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.383935 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:35.383970 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:35.386470 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:35.389714 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.389723 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.389738 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.390260 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.390633 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.390638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.390641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.390815 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.391339 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:35.391346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:35.391666 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 03:05:35.391897 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 03:05:35.392369 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":324,"threads_started":1}
I20260807 03:05:35.395100 25756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:05:35.408114 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.395834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37102)
0807 03:05:35.395986 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.396001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:35.396074 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:35.397045 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:35.397049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:35.397438 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:35.397559 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.397566 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.398887 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.398892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.399707 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.399718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.400458 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.400463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.400648 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.401163 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:35.401178 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:35.403075 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:35.405850 (+ 2775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.405855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.405860 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.406187 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.406543 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.406547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.406551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.406687 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.407228 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:35.407233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:35.407380 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 03:05:35.407493 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 03:05:35.407745 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 03:05:35.408224 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.396096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.5.122.61:36031)
0807 03:05:35.396679 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.396685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.396709 (+ 24us) server_negotiation.cc:407] Connection header received
0807 03:05:35.396747 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.396752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.396909 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.397029 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.397702 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.398752 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.399832 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.400322 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.400436 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.403259 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.403269 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.403280 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.403307 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.405737 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.406299 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.406305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.406308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.406447 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.406779 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.406784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.406788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.407109 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.407294 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.407658 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.407862 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":332,"threads_started":1}
I20260807 03:05:35.410290 25756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:05:35.411365 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.410963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52390)
0807 03:05:35.411117 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.411215 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 03:05:35.411581 25677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.412923 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.412624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52394)
0807 03:05:35.412745 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.412822 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 03:05:35.413127 25756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab.
Entry for principal kudu/127.5.122.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab.
I20260807 03:05:35.418278 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:45715
--webserver_interface=127.5.122.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab
--principal=kudu/127.5.122.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39451,127.5.122.61:36031,127.5.122.60:45715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:05:35.462423 25756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 1
I20260807 03:05:35.463577 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.463145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52398)
0807 03:05:35.463330 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.463431 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 03:05:35.464468 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.464143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52414)
0807 03:05:35.464278 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.464357 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 03:05:35.464658 25756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.532845 25756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 2
I20260807 03:05:35.533834 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.533416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52424)
0807 03:05:35.533599 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.533684 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 03:05:35.534788 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.534475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52436)
0807 03:05:35.534609 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.534674 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 03:05:35.534988 25756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.567955 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } attempt: 5
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:05:35.580947 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.568685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.5.122.61:36031)
0807 03:05:35.568865 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.568871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.569047 (+ 176us) server_negotiation.cc:407] Connection header received
0807 03:05:35.569222 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.569226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.569305 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.569404 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.570208 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.570945 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.572089 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.572507 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.572616 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.576007 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.576012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.576016 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.576047 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.578586 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.579320 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.579323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.579326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.579441 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.579805 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.579808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.579810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.580064 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.580172 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.580569 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.580735 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 03:05:35.581172 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.568684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37116)
0807 03:05:35.568956 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.568972 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:35.569101 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:35.569447 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:35.569452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:35.569861 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:35.570056 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.570065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.571078 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.571084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.571946 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.571959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.572645 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.572651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.572826 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.573382 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:35.573400 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:35.575829 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:35.578719 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.578725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.578729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.579224 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.579533 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.579538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.579542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.579709 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.580170 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:35.580175 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:35.580345 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 03:05:35.580506 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 03:05:35.580822 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 03:05:35.582125 25646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:05:35.583101 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.582743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52448)
0807 03:05:35.582903 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.582974 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 03:05:35.583925 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.583651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52458)
0807 03:05:35.583759 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.583834 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 03:05:35.584110 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.626181 25756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 3
I20260807 03:05:35.627351 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.626885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52474)
0807 03:05:35.627089 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.627182 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 03:05:35.628353 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.627950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52480)
0807 03:05:35.628169 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.628244 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260807 03:05:35.628556 25756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.635337 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 1
I20260807 03:05:35.636406 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.635921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52482)
0807 03:05:35.636115 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.636220 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 03:05:35.637334 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.637012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52484)
0807 03:05:35.637154 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.637221 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 03:05:35.637522 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.705720 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 2
I20260807 03:05:35.706956 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.706476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52488)
0807 03:05:35.706692 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.706796 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 03:05:35.707904 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.707565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52496)
0807 03:05:35.707712 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.707783 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 03:05:35.708097 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
W20260807 03:05:35.741433 25768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:35.741734 25768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:35.741870 25768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:35.748870 25768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:35.748979 25768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:35.749058 25768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:35.749133 25768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:35.749190 25768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:35.758687 25768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39451,127.5.122.61:36031,127.5.122.60:45715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab
--principal=kudu/127.5.122.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:45715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:35.759999 25768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:35.761255 25768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 03:05:35.771757 25756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 4
W20260807 03:05:35.772312 25774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:35.772492 25773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:35.772588 25776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:35.773145 25768 server_base.cc:1034] running on GCE node
I20260807 03:05:35.773299 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.772528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52500)
0807 03:05:35.773008 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.773107 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298}
I20260807 03:05:35.774058 25768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:35.774303 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.773956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52514)
0807 03:05:35.774120 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.774197 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 03:05:35.774569 25756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.775501 25768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:35.776696 25768 hybrid_clock.cc:648] HybridClock initialized: now 1786071935776644 us; error 40 us; skew 500 ppm
I20260807 03:05:35.777025 25768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:35.780061 25768 init.cc:377] Logged in from keytab as kudu/127.5.122.60@KRBTEST.COM (short username kudu)
I20260807 03:05:35.781448 25768 webserver.cc:460] Webserver started at http://127.5.122.60:33277/ using document root <none> and password file <none>
I20260807 03:05:35.782071 25768 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:35.782169 25768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:35.782467 25768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:35.785302 25768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data/instance:
uuid: "6f9a4214f29c43a796112c2cc1b473e6"
format_stamp: "Formatted at 2026-08-07 03:05:35 on dist-test-slave-fgb3"
server_key: "a96d7a76407590a711f58c072963466f"
server_key_iv: "7eb05bb29884cc14c28a7579c4efd9fd"
server_key_version: "encryptionkey@0"
I20260807 03:05:35.786062 25768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/wal/instance:
uuid: "6f9a4214f29c43a796112c2cc1b473e6"
format_stamp: "Formatted at 2026-08-07 03:05:35 on dist-test-slave-fgb3"
server_key: "a96d7a76407590a711f58c072963466f"
server_key_iv: "7eb05bb29884cc14c28a7579c4efd9fd"
server_key_version: "encryptionkey@0"
I20260807 03:05:35.791129 25768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:05:35.794272 25790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:35.795559 25768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:05:35.795708 25768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/wal
uuid: "6f9a4214f29c43a796112c2cc1b473e6"
format_stamp: "Formatted at 2026-08-07 03:05:35 on dist-test-slave-fgb3"
server_key: "a96d7a76407590a711f58c072963466f"
server_key_iv: "7eb05bb29884cc14c28a7579c4efd9fd"
server_key_version: "encryptionkey@0"
I20260807 03:05:35.795895 25768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:35.814276 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 3
I20260807 03:05:35.815290 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.814844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52520)
0807 03:05:35.815037 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.815130 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 03:05:35.816345 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.815957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52526)
0807 03:05:35.816153 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.816227 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 03:05:35.816573 25646 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:45715: Network error: Client connection negotiation failed: client connection to 127.5.122.60:45715: connect: Connection refused (error 111)
I20260807 03:05:35.817685 25768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:35.821815 25768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:35.822057 25768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:35.839380 25768 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.60:45715
I20260807 03:05:35.839416 25865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.60:45715 every 8 connection(s)
I20260807 03:05:35.840658 25768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
I20260807 03:05:35.841269 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 25768
I20260807 03:05:35.841467 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786071733698599-5608-0/minicluster-data/master-2/wal/instance
I20260807 03:05:35.842017 5608 external_mini_cluster.cc:1428] Setting key 8347505c6a5fba8d3bdfa62d03496c45
I20260807 03:05:35.845021 25866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:35.853120 25866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:35.865446 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.843859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52534 (local address 127.5.122.60:45715)
0807 03:05:35.844625 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.844636 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.844676 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:05:35.846300 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.846347 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.846748 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.847054 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.848867 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.850176 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.852163 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.852799 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.852939 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.857757 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.857767 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.857779 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.857826 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.861385 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.862147 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.862154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.862162 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.862409 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.862766 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.862772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.862776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.863308 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.863612 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.864046 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.864533 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":346,"threads_started":1}
I20260807 03:05:35.871791 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.855987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39451 (local address 127.0.0.1:60124)
0807 03:05:35.856723 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.856763 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:35.856924 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:35.857306 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:35.857311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:35.858100 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:35.858513 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.858526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.859622 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.859628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.861288 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.861303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.862219 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.862226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.862418 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.862969 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:35.862989 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:35.864940 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:35.869040 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.869051 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.869058 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.869493 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.869956 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.869961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.869964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.870116 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.870751 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:35.870759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:35.870943 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 03:05:35.871094 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 03:05:35.871391 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":378,"threads_started":1}
I20260807 03:05:35.871840 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.856129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.5.122.62:39451)
0807 03:05:35.856326 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.856333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.856862 (+ 529us) server_negotiation.cc:407] Connection header received
0807 03:05:35.857068 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.857074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.857160 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.857263 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.858650 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.859431 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.861411 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.862030 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.862142 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.865115 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.865119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.865123 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.865154 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.868925 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.869636 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.869642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.869645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.869803 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.870234 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.870239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.870243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.870598 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.870765 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.871253 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.871458 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:05:35.873198 25866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:05:35.884230 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.868421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.5.122.62:39451)
0807 03:05:35.869169 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.869174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.869196 (+ 22us) server_negotiation.cc:407] Connection header received
0807 03:05:35.869238 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.869242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.869304 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.869397 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.870951 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.872734 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.874221 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.874861 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.874979 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.878930 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.878936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.878941 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.878975 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.881697 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.882444 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.882448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.882450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.882557 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.882911 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.882914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.882916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.883168 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.883417 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.883693 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.883890 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":416,"threads_started":1}
I20260807 03:05:35.885928 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.873846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:36031 (local address 127.0.0.1:37128)
0807 03:05:35.874071 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.874088 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:35.874176 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:35.874534 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:35.874539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:35.874959 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:35.875125 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.875136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.876196 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.876201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.877037 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.877048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.877725 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.877731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.877933 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.878447 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:35.878462 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:35.880573 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:35.883695 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.883701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.883705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.884062 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.884487 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.884491 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.884493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.884637 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.885137 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:35.885142 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:35.885294 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 03:05:35.885402 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 03:05:35.885633 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 03:05:35.886114 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.873939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37128 (local address 127.5.122.61:36031)
0807 03:05:35.874106 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.874112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.874131 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:05:35.874297 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.874304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.874408 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.874509 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.875271 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.876065 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.877172 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.877585 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.877689 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.880744 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.880749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.880752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.880784 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.883581 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.884202 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.884208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.884211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.884382 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.884753 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.884756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.884758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.885008 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.885152 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.885521 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.885710 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:05:35.887431 25866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:05:35.900902 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.885877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37136 (local address 127.5.122.61:36031)
0807 03:05:35.886386 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.886391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.886410 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:05:35.886456 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.886462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.886528 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.886625 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.887996 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.888836 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.890237 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.890908 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.891038 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.895479 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.895486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.895491 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.895529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.897856 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.898844 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.898847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.898850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.898989 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.899446 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.899452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.899456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.899777 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.900001 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.900327 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.900551 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":8233,"threads_started":1}
I20260807 03:05:35.902237 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.888163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52536)
0807 03:05:35.888377 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.888394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:35.888484 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:35.889307 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:35.889312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:35.889781 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:35.889959 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.889970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.891254 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.891260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.892972 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.892990 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.893951 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.893958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.894157 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.894685 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:35.894703 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:35.896552 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:35.899657 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.899660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.899663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.900112 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.900518 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:35.900522 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:35.900525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:35.900670 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:35.901284 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:35.901288 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:35.901429 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 03:05:35.901583 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 03:05:35.901876 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 03:05:35.902369 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.888217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52536 (local address 127.5.122.60:45715)
0807 03:05:35.889055 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.889060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.889076 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:05:35.889119 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.889123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.889186 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.889280 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.890110 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.891123 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.893136 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.893765 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.893890 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.896715 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.896721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.896726 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.896759 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.899553 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.900217 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.900222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.900225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.900366 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.900777 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.900781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.900784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.901092 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.901207 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.901680 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.902069 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":725}
I20260807 03:05:35.905647 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:05:35.907272 25866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: Bootstrap starting.
I20260807 03:05:35.910367 25866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:35.911342 25866 log.cc:824] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:35.915998 25866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: No bootstrap required, opened a new log
I20260807 03:05:35.926591 25866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:35.927142 25866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:35.927321 25866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9a4214f29c43a796112c2cc1b473e6, State: Initialized, Role: FOLLOWER
I20260807 03:05:35.928025 25866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:35.931073 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.912136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.5.122.62:39451)
0807 03:05:35.912324 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.912331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.912912 (+ 581us) server_negotiation.cc:407] Connection header received
0807 03:05:35.913700 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.913706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.913804 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.913910 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.916127 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.917159 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.919512 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.920150 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.920247 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.924143 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.924150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.924155 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.924194 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.927525 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.928532 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.928537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.928541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.928691 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.929168 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.929174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.929178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.929547 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.929825 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.930424 (+ 599us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.930656 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:05:35.931883 25883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:35.932281 25883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:05:35.933195 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.911828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.5.122.61:36031)
0807 03:05:35.911999 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.912361 (+ 362us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.913426 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 03:05:35.913818 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.913824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.913902 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.913982 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.916553 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.917696 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.920161 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.920846 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.920937 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.925233 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.925239 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.925243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.925281 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.928817 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.929692 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.929697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.929700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.929868 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.930404 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.930409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.930412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.930794 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.931027 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.932635 (+ 1608us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.932860 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:05:35.933486 25866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:35.940860 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.913277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.5.122.60:45715)
0807 03:05:35.913469 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.913475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.924396 (+ 10921us) server_negotiation.cc:407] Connection header received
0807 03:05:35.924815 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.924821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.924895 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.924967 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.926626 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.927675 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.929692 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.930351 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.930438 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.934024 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:35.934030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:35.934034 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:35.934070 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:35.937864 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.938672 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.938677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.938680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.938884 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:35.939309 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:35.939315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:35.939318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:35.939705 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:35.939964 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:35.940276 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 03:05:35.940499 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:05:35.940348 25768 master_runner.cc:186] Error getting master registration for 127.5.122.62:39451: OK, instance_id { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" instance_seqno: 1786071934861773 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 03:05:35.948796 25768 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 03:05:35.949008 25898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 03:05:35.949127 25898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:05:35.993762 25646 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 4
Aug 07 03:05:35 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071934, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:05:36.007206 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.994391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.5.122.60:45715)
0807 03:05:35.994549 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:35.994554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:35.994571 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:05:35.994748 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:35.994753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:35.994816 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:35.994893 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:35.995640 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.996424 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.997609 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.998092 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.998207 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:36.001681 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:36.001687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:36.001691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:36.001727 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:36.004638 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:36.005249 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:36.005254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:36.005256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:36.005407 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:36.005763 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:36.005766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:36.005769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:36.006052 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:36.006193 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:36.006613 (+ 420us) server_negotiation.cc:299] Negotiation successful
0807 03:05:36.006848 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:05:36.007210 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:35.994301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52562)
0807 03:05:35.994509 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:35.994528 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:35.994634 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:35.994951 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:35.994956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:35.995337 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:35.995494 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.995503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.996551 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.996557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.997472 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:35.997484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:35.998231 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:35.998237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:35.998438 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:35.999024 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:35.999040 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:36.001483 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:36.004767 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:36.004772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:36.004776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:36.005140 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:36.005521 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:36.005526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:36.005531 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:36.005661 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:36.006168 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:36.006172 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:36.006360 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 03:05:36.006501 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 03:05:36.006823 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 03:05:36.011439 25646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458: Bootstrap starting.
I20260807 03:05:36.014474 25646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:36.015508 25646 log.cc:824] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:36.019697 25646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458: No bootstrap required, opened a new log
I20260807 03:05:36.026636 25646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:36.026938 25646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:36.027046 25646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876f3a8cad52454fa4f292aa96378458, State: Initialized, Role: FOLLOWER
I20260807 03:05:36.027571 25646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:36.028442 25901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.028823 25901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:05:36.029615 25646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:36.046177 25901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 03:05:36.046233 25915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 03:05:36.046547 25901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:36.048188 25901 leader_election.cc:288] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91d58966d3494b20896be6b88da04f37 (127.5.122.61:36031), 6f9a4214f29c43a796112c2cc1b473e6 (127.5.122.60:45715)
I20260807 03:05:36.051667 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "876f3a8cad52454fa4f292aa96378458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9a4214f29c43a796112c2cc1b473e6" is_pre_election: true
I20260807 03:05:36.052305 25829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 876f3a8cad52454fa4f292aa96378458 in term 0.
I20260807 03:05:36.052981 25565 leader_election.cc:302] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9a4214f29c43a796112c2cc1b473e6, 876f3a8cad52454fa4f292aa96378458; no voters:
I20260807 03:05:36.053395 25901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:05:36.053550 25901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:05:36.053659 25901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:36.057245 25901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
W20260807 03:05:36.057747 25915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:05:36.059052 25901 leader_election.cc:288] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [CANDIDATE]: Term 1 election: Requested vote from peers 91d58966d3494b20896be6b88da04f37 (127.5.122.61:36031), 6f9a4214f29c43a796112c2cc1b473e6 (127.5.122.60:45715)
I20260807 03:05:36.059140 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "876f3a8cad52454fa4f292aa96378458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9a4214f29c43a796112c2cc1b473e6"
I20260807 03:05:36.059340 25829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:36.064021 25829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 876f3a8cad52454fa4f292aa96378458 in term 1.
I20260807 03:05:36.064574 25565 leader_election.cc:302] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9a4214f29c43a796112c2cc1b473e6, 876f3a8cad52454fa4f292aa96378458; no voters:
I20260807 03:05:36.064980 25901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:05:36.044943 25540 master_runner.cc:186] Error getting master registration for 127.5.122.61:36031: OK, instance_id { permanent_uuid: "91d58966d3494b20896be6b88da04f37" instance_seqno: 1786071935352831 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 03:05:36.069384 25540 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 03:05:36.069806 25901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [term 1 LEADER]: Becoming Leader. State: Replica: 876f3a8cad52454fa4f292aa96378458, State: Running, Role: LEADER
I20260807 03:05:36.070555 25901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:36.073747 25756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } attempt: 5
W20260807 03:05:36.074115 25803 tablet.cc:2369] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:05:36.074126 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 876f3a8cad52454fa4f292aa96378458. Latest consensus state: current_term: 1 leader_uuid: "876f3a8cad52454fa4f292aa96378458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.074504 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: This master's current role is: LEADER
Aug 07 03:05:36 dist-test-slave-fgb3 krb5kdc[25522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071935, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:05:36.088754 25920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:05:36.091320 25920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:05:36.101037 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:36.074291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:45715 (local address 127.0.0.1:52572)
0807 03:05:36.074669 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:36.074688 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:36.074801 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:36.076832 (+ 2031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:36.076837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:36.077341 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:36.077559 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:36.077573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:36.079083 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:36.079088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:36.080469 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:36.080485 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:36.081751 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:36.081757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:36.081971 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:36.082718 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:36.082771 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:36.087213 (+ 4442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:36.095443 (+ 8230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:36.095476 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:36.095481 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:36.096433 (+ 952us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:36.097319 (+ 886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:36.097324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:36.097328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:36.097647 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:36.099062 (+ 1415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:36.099095 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:36.099387 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 03:05:36.099660 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 03:05:36.100195 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255}
W20260807 03:05:36.102432 25573 tablet.cc:2369] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:05:36.103487 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:36.075772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.5.122.60:45715)
0807 03:05:36.076091 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:36.076100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:36.076118 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:05:36.076171 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:36.076176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:36.076247 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:36.076341 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:36.077741 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:36.078920 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:36.080696 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:36.081374 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:36.081499 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:36.087519 (+ 6020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:36.087527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:36.087532 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:36.087573 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:36.095195 (+ 7622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:36.096761 (+ 1566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:36.096766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:36.096770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:36.097102 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:36.097932 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:36.097936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:36.097940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:36.098828 (+ 888us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:36.101115 (+ 2287us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:36.102114 (+ 999us) server_negotiation.cc:299] Negotiation successful
0807 03:05:36.102705 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260807 03:05:36.109449 25756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37: Bootstrap starting.
I20260807 03:05:36.113343 25756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:36.113409 25829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [term 1 FOLLOWER]: Refusing update from remote peer 876f3a8cad52454fa4f292aa96378458: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:36.114683 25756 log.cc:824] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:36.115445 25918 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:05:36.126461 25756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37: No bootstrap required, opened a new log
I20260807 03:05:36.137136 25756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:36.137570 25756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:36.137704 25756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d58966d3494b20896be6b88da04f37, State: Initialized, Role: FOLLOWER
I20260807 03:05:36.139672 25883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 876f3a8cad52454fa4f292aa96378458. Latest consensus state: current_term: 1 leader_uuid: "876f3a8cad52454fa4f292aa96378458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.140005 25883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:05:36.140633 25756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } }
I20260807 03:05:36.143671 25921 mvcc.cc:205] Tried to move back new op lower bound from 7315750650304311296 to 7315750650149724160. Current Snapshot: MvccSnapshot[applied={T|T < 7315750650304311296}]
I20260807 03:05:36.144395 25924 mvcc.cc:205] Tried to move back new op lower bound from 7315750650304311296 to 7315750650149724160. Current Snapshot: MvccSnapshot[applied={T|T < 7315750650304311296}]
I20260807 03:05:36.156795 25920 catalog_manager.cc:1347] Generated new cluster ID: 0b1d961a635749b4a7c8dccde12693ae
I20260807 03:05:36.156905 25920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:05:36.158208 25883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "876f3a8cad52454fa4f292aa96378458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.158505 25883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:05:36.159763 25927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.160168 25927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:05:36.161877 25756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:36.164853 25714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "876f3a8cad52454fa4f292aa96378458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91d58966d3494b20896be6b88da04f37" is_pre_election: true
I20260807 03:05:36.165472 25714 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 876f3a8cad52454fa4f292aa96378458 in term 0.
I20260807 03:05:36.166798 25707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:36.166754 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "876f3a8cad52454fa4f292aa96378458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.166997 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:36.171332 25713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "876f3a8cad52454fa4f292aa96378458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91d58966d3494b20896be6b88da04f37"
I20260807 03:05:36.173098 25707 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [term 1 FOLLOWER]: Refusing update from remote peer 876f3a8cad52454fa4f292aa96378458: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:36.173851 25901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:05:36.179390 25920 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:05:36.180985 25920 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:05:36.200567 25927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 876f3a8cad52454fa4f292aa96378458. Latest consensus state: current_term: 1 leader_uuid: "876f3a8cad52454fa4f292aa96378458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.200896 25927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:05:36.201375 25939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "876f3a8cad52454fa4f292aa96378458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.202710 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "876f3a8cad52454fa4f292aa96378458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876f3a8cad52454fa4f292aa96378458" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39451 } } peers { permanent_uuid: "91d58966d3494b20896be6b88da04f37" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 36031 } } peers { permanent_uuid: "6f9a4214f29c43a796112c2cc1b473e6" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 45715 } } }
I20260807 03:05:36.202982 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:36.204457 25920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 876f3a8cad52454fa4f292aa96378458: Generated new TSK 0
I20260807 03:05:36.205399 25920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:05:36.210112 25939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:05:36.325618 25968 catalog_manager.cc:1259] Loaded cluster ID: 0b1d961a635749b4a7c8dccde12693ae
I20260807 03:05:36.325767 25968 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37: loading cluster ID for follower catalog manager: success
I20260807 03:05:36.329278 25968 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37: acquiring CA information for follower catalog manager: success
I20260807 03:05:36.331866 25968 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91d58966d3494b20896be6b88da04f37: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 03:05:36.951090 25898 catalog_manager.cc:1259] Loaded cluster ID: 0b1d961a635749b4a7c8dccde12693ae
I20260807 03:05:36.951191 25898 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: loading cluster ID for follower catalog manager: success
I20260807 03:05:36.953497 25898 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: acquiring CA information for follower catalog manager: success
I20260807 03:05:36.955154 25898 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f9a4214f29c43a796112c2cc1b473e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 03:05:37.361006 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 25540
I20260807 03:05:37.362524 25638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:37.471856 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 25540
I20260807 03:05:37.497165 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 25655
I20260807 03:05:37.498261 25748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:37.603926 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 25655
I20260807 03:05:37.628574 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 25768
I20260807 03:05:37.629699 25860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:37.737918 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 25768
2026-08-07T03:05:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4734 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:05:37 dist-test-slave-fgb3 krb5kdc[25976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:05:37 dist-test-slave-fgb3 krb5kdc[25976](info): set up 2 sockets
Aug 07 03:05:37 dist-test-slave-fgb3 krb5kdc[25976](info): commencing operation
krb5kdc: starting...
W20260807 03:05:38.916936 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:05:39 dist-test-slave-fgb3 krb5kdc[25976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:05:39.166769 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:36479
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:36479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:39.482664 26000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:39.482928 26000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:39.483057 26000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:39.489840 26000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:39.489930 26000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:39.490001 26000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:39.490075 26000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:39.490134 26000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:39.499301 26000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:36479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:36479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:39.500391 26000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:39.501554 26000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:39.512084 26007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:39.512130 26008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:39.512149 26011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:39.513130 26000 server_base.cc:1034] running on GCE node
I20260807 03:05:39.513794 26000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:39.515043 26000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:39.516238 26000 hybrid_clock.cc:648] HybridClock initialized: now 1786071939516194 us; error 33 us; skew 500 ppm
I20260807 03:05:39.516551 26000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:39 dist-test-slave-fgb3 krb5kdc[25976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:39.524752 26000 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:05:39.526119 26000 webserver.cc:460] Webserver started at http://127.5.122.62:42659/ using document root <none> and password file <none>
I20260807 03:05:39.526777 26000 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:39.526892 26000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:39.527180 26000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:39.529939 26000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "0c42b48a21b441daaf8396c0249d9732"
format_stamp: "Formatted at 2026-08-07 03:05:39 on dist-test-slave-fgb3"
server_key: "202bd460dd5f26f9a20cfe7ef3f9c6f8"
server_key_iv: "c31337e3a6b76daedd4ed6f860b76718"
server_key_version: "encryptionkey@0"
I20260807 03:05:39.530682 26000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "0c42b48a21b441daaf8396c0249d9732"
format_stamp: "Formatted at 2026-08-07 03:05:39 on dist-test-slave-fgb3"
server_key: "202bd460dd5f26f9a20cfe7ef3f9c6f8"
server_key_iv: "c31337e3a6b76daedd4ed6f860b76718"
server_key_version: "encryptionkey@0"
I20260807 03:05:39.535548 26000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:05:39.538715 26022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:39.539949 26000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 03:05:39.540099 26000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "0c42b48a21b441daaf8396c0249d9732"
format_stamp: "Formatted at 2026-08-07 03:05:39 on dist-test-slave-fgb3"
server_key: "202bd460dd5f26f9a20cfe7ef3f9c6f8"
server_key_iv: "c31337e3a6b76daedd4ed6f860b76718"
server_key_version: "encryptionkey@0"
I20260807 03:05:39.540267 26000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:39.564884 26000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:39.568928 26000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:39.569186 26000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:39.585958 26000 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:36479
I20260807 03:05:39.586014 26093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:36479 every 8 connection(s)
I20260807 03:05:39.587321 26000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:05:39.590831 26094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:39.596067 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 26000
I20260807 03:05:39.596282 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:05:39.597100 5608 external_mini_cluster.cc:1428] Setting key 0a01fe4af7750cd38826d454d9d3ecd2
I20260807 03:05:39.601572 26094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732: Bootstrap starting.
I20260807 03:05:39.604920 26094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:39.606045 26094 log.cc:824] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:39.608901 26094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732: No bootstrap required, opened a new log
Aug 07 03:05:39 dist-test-slave-fgb3 krb5kdc[25976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:39.616562 26094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c42b48a21b441daaf8396c0249d9732" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36479 } }
I20260807 03:05:39.616927 26094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:39.617023 26094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c42b48a21b441daaf8396c0249d9732, State: Initialized, Role: FOLLOWER
I20260807 03:05:39.617574 26094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c42b48a21b441daaf8396c0249d9732" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36479 } }
I20260807 03:05:39.617815 26094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:05:39.617964 26094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:05:39.618165 26094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:39.619247 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:39.599034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.5.122.62:36479)
0807 03:05:39.599853 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:39.599864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:39.599910 (+ 46us) server_negotiation.cc:407] Connection header received
0807 03:05:39.601573 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:39.601594 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:39.601969 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:39.602262 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:39.604030 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:39.605315 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:39.607066 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:39.607729 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:39.607859 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:39.612359 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:39.612367 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:39.612380 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:39.612415 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:39.615334 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:39.615975 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:39.615981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:39.615986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:39.616144 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:39.616520 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:39.616523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:39.616525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:39.617073 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:39.617352 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:39.617791 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 03:05:39.618207 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":391,"threads_started":1}
I20260807 03:05:39.620407 26094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c42b48a21b441daaf8396c0249d9732" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36479 } }
I20260807 03:05:39.620859 26094 leader_election.cc:302] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c42b48a21b441daaf8396c0249d9732; no voters:
I20260807 03:05:39.621460 26094 leader_election.cc:288] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:05:39.621552 26102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:05:39.622767 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:05:39.622820 26102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [term 1 LEADER]: Becoming Leader. State: Replica: 0c42b48a21b441daaf8396c0249d9732, State: Running, Role: LEADER
I20260807 03:05:39.623365 26102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c42b48a21b441daaf8396c0249d9732" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36479 } }
I20260807 03:05:39.623566 26094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:39.626084 26103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c42b48a21b441daaf8396c0249d9732" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c42b48a21b441daaf8396c0249d9732" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36479 } } }
I20260807 03:05:39.626510 26103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:39.626515 26105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c42b48a21b441daaf8396c0249d9732. Latest consensus state: current_term: 1 leader_uuid: "0c42b48a21b441daaf8396c0249d9732" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c42b48a21b441daaf8396c0249d9732" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36479 } } }
I20260807 03:05:39.627287 26105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:39.627512 26111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:05:39.631904 26111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:05:39.639629 26111 catalog_manager.cc:1347] Generated new cluster ID: 7f3bc15c5025456a85b108e177a283a0
I20260807 03:05:39.639729 26111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:05:39.662046 26111 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:05:39.663214 26111 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:05:39.673601 26111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c42b48a21b441daaf8396c0249d9732: Generated new TSK 0
I20260807 03:05:39.674252 26111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:05:39.683187 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 26000
I20260807 03:05:39.684175 26089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:39.774191 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 26000
2026-08-07T03:05:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2030 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:05:39 dist-test-slave-fgb3 krb5kdc[26135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:05:39 dist-test-slave-fgb3 krb5kdc[26135](info): set up 2 sockets
Aug 07 03:05:39 dist-test-slave-fgb3 krb5kdc[26135](info): commencing operation
krb5kdc: starting...
W20260807 03:05:40.957002 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:05:41 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:05:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:05:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:05:41.191788 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39491
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:41.514070 26150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:41.514344 26150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:41.514477 26150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:41.521512 26150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:41.521615 26150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:41.521695 26150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:41.521766 26150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:41.521827 26150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:41.531378 26150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:41.532598 26150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:41.533847 26150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:41.544715 26158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:41.544804 26166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:41.544878 26160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:41.545783 26150 server_base.cc:1034] running on GCE node
I20260807 03:05:41.546535 26150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:41.547742 26150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:41.548933 26150 hybrid_clock.cc:648] HybridClock initialized: now 1786071941548885 us; error 36 us; skew 500 ppm
I20260807 03:05:41.549239 26150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:41 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:41.552050 26150 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:05:41.553361 26150 webserver.cc:460] Webserver started at http://127.5.122.62:36755/ using document root <none> and password file <none>
I20260807 03:05:41.553962 26150 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:41.554076 26150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:41.554402 26150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:41.557233 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29"
format_stamp: "Formatted at 2026-08-07 03:05:41 on dist-test-slave-fgb3"
server_key: "60da7541437f834baa0827496c426e80"
server_key_iv: "503ac9110cb40d1176bcc96fdd7f6770"
server_key_version: "encryptionkey@0"
I20260807 03:05:41.557940 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29"
format_stamp: "Formatted at 2026-08-07 03:05:41 on dist-test-slave-fgb3"
server_key: "60da7541437f834baa0827496c426e80"
server_key_iv: "503ac9110cb40d1176bcc96fdd7f6770"
server_key_version: "encryptionkey@0"
I20260807 03:05:41.562695 26150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260807 03:05:41.565824 26178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:41.566996 26150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:05:41.567147 26150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29"
format_stamp: "Formatted at 2026-08-07 03:05:41 on dist-test-slave-fgb3"
server_key: "60da7541437f834baa0827496c426e80"
server_key_iv: "503ac9110cb40d1176bcc96fdd7f6770"
server_key_version: "encryptionkey@0"
I20260807 03:05:41.567307 26150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:41.584726 26150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:41.588399 26150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:41.588650 26150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:41.605154 26150 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:39491
I20260807 03:05:41.605201 26257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:39491 every 8 connection(s)
I20260807 03:05:41.606503 26150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:05:41.609894 26258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:41.616601 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 26150
I20260807 03:05:41.616822 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:05:41.617497 5608 external_mini_cluster.cc:1428] Setting key 4af05f6b6955a96180220d63466844aa
I20260807 03:05:41.619765 26258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Bootstrap starting.
I20260807 03:05:41.622819 26258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:41.624090 26258 log.cc:824] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:41.627192 26258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: No bootstrap required, opened a new log
Aug 07 03:05:41 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:41.636962 26258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:41.637470 26258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:41.637590 26258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0d7bd5b2ba4d00ab6615b6ed9caa29, State: Initialized, Role: FOLLOWER
I20260807 03:05:41.638401 26258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:41.638690 26258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:05:41.638826 26258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:05:41.639065 26258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:41.642565 26258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:41.643294 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:41.619524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.5.122.62:39491)
0807 03:05:41.620364 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:41.620375 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:41.620419 (+ 44us) server_negotiation.cc:407] Connection header received
0807 03:05:41.622218 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:41.622248 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:41.622662 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:41.623007 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:41.624863 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:41.626479 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:41.628281 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:41.628961 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:41.629098 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:41.634074 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:41.634086 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:41.634100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:41.634143 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:41.638086 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:41.639006 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:41.639022 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:41.639032 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:41.639257 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:41.639706 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:41.639712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:41.639715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:41.640283 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:41.640590 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:41.641596 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0807 03:05:41.641982 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":425,"threads_started":1}
I20260807 03:05:41.643290 26258 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0d7bd5b2ba4d00ab6615b6ed9caa29; no voters:
I20260807 03:05:41.646068 26258 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:05:41.646201 26264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:05:41.648710 26264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 LEADER]: Becoming Leader. State: Replica: 1b0d7bd5b2ba4d00ab6615b6ed9caa29, State: Running, Role: LEADER
I20260807 03:05:41.648762 26258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:41.649276 26264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:41.654683 26266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0d7bd5b2ba4d00ab6615b6ed9caa29. Latest consensus state: current_term: 1 leader_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } } }
I20260807 03:05:41.655098 26266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:41.661324 26265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } } }
I20260807 03:05:41.661697 26265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: This master's current role is: LEADER
W20260807 03:05:41.665673 26284 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 03:05:41.665779 26284 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:05:41.665906 26276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:05:41.668376 26276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:05:41.677742 26276 catalog_manager.cc:1347] Generated new cluster ID: cca53674857548b7b8b847b5e28a8b23
I20260807 03:05:41.677850 26276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:05:41.693511 26276 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:05:41.695611 26276 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:05:41.707612 26276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Generated new TSK 0
I20260807 03:05:41.708745 26276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:05:41.770994 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:39491
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:42.088909 26292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:42.089174 26292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:42.089306 26292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:42.096367 26292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:42.096470 26292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:42.096623 26292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:05:42.105981 26292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:42.107259 26292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:42.108512 26292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:42.120728 26299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:42.120688 26300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:42.120775 26304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:42.121798 26292 server_base.cc:1034] running on GCE node
I20260807 03:05:42.122401 26292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:42.123281 26292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:42.124480 26292 hybrid_clock.cc:648] HybridClock initialized: now 1786071942124422 us; error 49 us; skew 500 ppm
I20260807 03:05:42.124779 26292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:42 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071942, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:42.127915 26292 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:05:42.129336 26292 webserver.cc:460] Webserver started at http://127.5.122.1:38383/ using document root <none> and password file <none>
I20260807 03:05:42.130010 26292 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:42.130118 26292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:42.130455 26292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:42.133337 26292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "134a1de9819747159579124934a0a554"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "216fdeb6c4c828327f54d7277475c15b"
server_key_iv: "9c27b1c09dd83b704ce3e31f0158de99"
server_key_version: "encryptionkey@0"
I20260807 03:05:42.134083 26292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "134a1de9819747159579124934a0a554"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "216fdeb6c4c828327f54d7277475c15b"
server_key_iv: "9c27b1c09dd83b704ce3e31f0158de99"
server_key_version: "encryptionkey@0"
I20260807 03:05:42.139052 26292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 03:05:42.142448 26314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:42.143668 26292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:05:42.143808 26292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "134a1de9819747159579124934a0a554"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "216fdeb6c4c828327f54d7277475c15b"
server_key_iv: "9c27b1c09dd83b704ce3e31f0158de99"
server_key_version: "encryptionkey@0"
I20260807 03:05:42.144009 26292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:42.162446 26292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:42.166399 26292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:42.166662 26292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:42.167503 26292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:42.168766 26292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:42.168839 26292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:42.168957 26292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:42.169005 26292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:42.196988 26292 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:44881
I20260807 03:05:42.197049 26485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:44881 every 8 connection(s)
I20260807 03:05:42.198146 26292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:05:42.205066 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 26292
I20260807 03:05:42.205255 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:05:42.205888 5608 external_mini_cluster.cc:1428] Setting key 0b45f49ceee20218557efd0d5e5feb71
Aug 07 03:05:42 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071942, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:42.217639 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:42.200482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:47465 (local address 127.5.122.62:39491)
0807 03:05:42.200910 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:42.200914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:42.201721 (+ 807us) server_negotiation.cc:407] Connection header received
0807 03:05:42.203472 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:42.203478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:42.203557 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:42.203649 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:42.205039 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.205837 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.207702 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.208275 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.208407 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:42.211706 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:42.211712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:42.211716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:42.211751 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:42.214701 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:42.215448 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:42.215453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:42.215456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:42.215592 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:42.215974 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:42.215979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:42.215981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:42.216293 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:42.216427 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:42.217097 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 03:05:42.217306 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":233,"threads_started":1}
I20260807 03:05:42.218623 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:42.200908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:47465)
0807 03:05:42.201568 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:42.201603 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:42.203215 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:42.203881 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:42.203889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:42.204354 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:42.204871 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:42.204888 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.205985 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.205989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:42.207564 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:42.207578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.208488 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.208494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:42.208676 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:42.209286 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:42.209313 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:42.211480 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:42.214821 (+ 3341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:42.214829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:42.214842 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:42.215318 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:42.215681 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:42.215685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:42.215688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:42.215873 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:42.216403 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:42.216410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:42.216746 (+ 336us) client_negotiation.cc:769] Sending connection context
0807 03:05:42.216983 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 03:05:42.217398 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":262,"threads_started":1}
I20260807 03:05:42.219969 26487 heartbeater.cc:347] Connected to a master server at 127.5.122.62:39491
I20260807 03:05:42.220273 26487 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:42.221019 26487 heartbeater.cc:511] Master 127.5.122.62:39491 requested a full tablet report, sending...
I20260807 03:05:42.223047 26199 ts_manager.cc:194] Registered new tserver with Master: 134a1de9819747159579124934a0a554 (127.5.122.1:44881)
I20260807 03:05:42.224382 26199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:47465
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:05:42.262914 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:39491
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:42.581823 26498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:42.582075 26498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:42.582206 26498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:42.589131 26498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:42.589229 26498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:42.589394 26498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:05:42.598649 26498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:42.599785 26498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:42.601016 26498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:42.620126 26506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:42.620790 26513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:42.621034 26507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:42.622263 26498 server_base.cc:1034] running on GCE node
I20260807 03:05:42.622869 26498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:42.623729 26498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:42.624934 26498 hybrid_clock.cc:648] HybridClock initialized: now 1786071942624884 us; error 37 us; skew 500 ppm
I20260807 03:05:42.625231 26498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:42 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071942, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:42.628373 26498 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:05:42.629748 26498 webserver.cc:460] Webserver started at http://127.5.122.2:34873/ using document root <none> and password file <none>
I20260807 03:05:42.630450 26498 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:42.630568 26498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:42.630839 26498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:42.633601 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "b748f970ed9f494daf27b1cabcafcdc3"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "d95113664e19273dc25b2e299c476303"
server_key_iv: "1cddd782e4818fe9a33a864300e52259"
server_key_version: "encryptionkey@0"
I20260807 03:05:42.634334 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "b748f970ed9f494daf27b1cabcafcdc3"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "d95113664e19273dc25b2e299c476303"
server_key_iv: "1cddd782e4818fe9a33a864300e52259"
server_key_version: "encryptionkey@0"
I20260807 03:05:42.639047 26498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 03:05:42.642235 26525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:42.643537 26498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:05:42.643671 26498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "b748f970ed9f494daf27b1cabcafcdc3"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "d95113664e19273dc25b2e299c476303"
server_key_iv: "1cddd782e4818fe9a33a864300e52259"
server_key_version: "encryptionkey@0"
I20260807 03:05:42.643843 26498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:42.658632 26498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:42.662487 26498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:42.662714 26498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:42.663539 26498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:42.664737 26498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:42.664811 26498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:42.664902 26498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:42.664944 26498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:42.698173 26498 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:39527
I20260807 03:05:42.698190 26700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:39527 every 8 connection(s)
I20260807 03:05:42.699589 26498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:05:42.702941 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 26498
I20260807 03:05:42.703146 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:05:42.703763 5608 external_mini_cluster.cc:1428] Setting key f37b394c64330d17e8710403b66d4929
Aug 07 03:05:42 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071942, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:42.720383 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:42.702006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:47469 (local address 127.5.122.62:39491)
0807 03:05:42.702183 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:42.702188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:42.703547 (+ 1359us) server_negotiation.cc:407] Connection header received
0807 03:05:42.705685 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:42.705691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:42.705782 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:42.705876 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:42.707379 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.708482 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.710006 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.710623 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.710763 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:42.714641 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:42.714647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:42.714651 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:42.714685 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:42.717435 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:42.718126 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:42.718130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:42.718132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:42.718261 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:42.718753 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:42.718757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:42.718759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:42.719051 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:42.719177 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:42.719985 (+ 808us) server_negotiation.cc:299] Negotiation successful
0807 03:05:42.720159 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 03:05:42.721624 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:42.702563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:47469)
0807 03:05:42.703398 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:42.703442 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:42.705385 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:42.706113 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:42.706121 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:42.706669 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:42.707205 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:42.707221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.708637 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.708641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:42.709857 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:42.709869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:42.710815 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:42.710820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:42.711014 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:42.711699 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:42.711723 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:42.714417 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:42.717560 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:42.717570 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:42.717585 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:42.718005 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:42.718379 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:42.718384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:42.718387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:42.718616 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:42.719183 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:42.719191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:42.719569 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 03:05:42.719846 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 03:05:42.720330 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":346,"threads_started":1}
I20260807 03:05:42.722977 26701 heartbeater.cc:347] Connected to a master server at 127.5.122.62:39491
I20260807 03:05:42.723274 26701 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:42.724148 26701 heartbeater.cc:511] Master 127.5.122.62:39491 requested a full tablet report, sending...
I20260807 03:05:42.725636 26199 ts_manager.cc:194] Registered new tserver with Master: b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527)
I20260807 03:05:42.726698 26199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:47469
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:05:42.763307 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:39491
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:43.075336 26712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:43.075591 26712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:43.075721 26712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:43.083019 26712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:43.083129 26712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:43.083293 26712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:05:43.092795 26712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:43.094112 26712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:43.095415 26712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:43.107273 26720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:43.107318 26725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:43.107245 26723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:43.108081 26712 server_base.cc:1034] running on GCE node
I20260807 03:05:43.108655 26712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:43.109508 26712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:43.110698 26712 hybrid_clock.cc:648] HybridClock initialized: now 1786071943110649 us; error 39 us; skew 500 ppm
I20260807 03:05:43.110976 26712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:43 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071943, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:43.113935 26712 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:05:43.115279 26712 webserver.cc:460] Webserver started at http://127.5.122.3:41043/ using document root <none> and password file <none>
I20260807 03:05:43.115932 26712 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:43.116037 26712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:43.116281 26712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:05:43.119199 26712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "749cb52b4c0d40b2bacd3e647cc227b9"
format_stamp: "Formatted at 2026-08-07 03:05:43 on dist-test-slave-fgb3"
server_key: "85f75fd20c436ddaebb50393ffc19ebf"
server_key_iv: "5aa0a4fef66ae5061d2ebdb3022bf787"
server_key_version: "encryptionkey@0"
I20260807 03:05:43.119884 26712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "749cb52b4c0d40b2bacd3e647cc227b9"
format_stamp: "Formatted at 2026-08-07 03:05:43 on dist-test-slave-fgb3"
server_key: "85f75fd20c436ddaebb50393ffc19ebf"
server_key_iv: "5aa0a4fef66ae5061d2ebdb3022bf787"
server_key_version: "encryptionkey@0"
I20260807 03:05:43.124708 26712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:05:43.127902 26735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:43.129117 26712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 03:05:43.129240 26712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "749cb52b4c0d40b2bacd3e647cc227b9"
format_stamp: "Formatted at 2026-08-07 03:05:43 on dist-test-slave-fgb3"
server_key: "85f75fd20c436ddaebb50393ffc19ebf"
server_key_iv: "5aa0a4fef66ae5061d2ebdb3022bf787"
server_key_version: "encryptionkey@0"
I20260807 03:05:43.129390 26712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:43.144810 26712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:43.149077 26712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:43.149325 26712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:43.150238 26712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:43.151515 26712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:43.151585 26712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:43.151675 26712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:43.151717 26712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:43.181123 26712 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36627
I20260807 03:05:43.181162 26897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36627 every 8 connection(s)
I20260807 03:05:43.182350 26712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:05:43.183048 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 26712
I20260807 03:05:43.183228 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:05:43.183769 5608 external_mini_cluster.cc:1428] Setting key afdd75f8266947f0c19f29b9d5ebb495
Aug 07 03:05:43 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071943, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:43.201848 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.185064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:43017 (local address 127.5.122.62:39491)
0807 03:05:43.185235 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.185240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.186563 (+ 1323us) server_negotiation.cc:407] Connection header received
0807 03:05:43.188447 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.188453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.188555 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.188642 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:43.190068 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.190865 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.192350 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.192744 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.192849 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.195982 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:43.195986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:43.195990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:43.196019 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:43.199051 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.199743 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.199748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.199753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.199909 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.200293 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.200298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.200302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.200570 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:43.200701 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.201310 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.201525 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 03:05:43.202705 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.185637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:43017)
0807 03:05:43.186430 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.186471 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:43.188199 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:43.188886 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:43.188895 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:43.189358 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:43.189912 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.189930 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.191015 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.191019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.192213 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.192226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.192962 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.192970 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.193186 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.193739 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:43.193759 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:43.195811 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:43.199162 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:43.199172 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:43.199185 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:43.199635 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:43.200011 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:43.200014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:43.200017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:43.200189 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:43.200692 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:43.200699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:43.200987 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 03:05:43.201215 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 03:05:43.201685 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":310,"threads_started":1}
I20260807 03:05:43.203820 26898 heartbeater.cc:347] Connected to a master server at 127.5.122.62:39491
I20260807 03:05:43.204082 26898 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:43.204743 26898 heartbeater.cc:511] Master 127.5.122.62:39491 requested a full tablet report, sending...
I20260807 03:05:43.206038 26199 ts_manager.cc:194] Registered new tserver with Master: 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
I20260807 03:05:43.206869 26199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:43017
I20260807 03:05:43.211427 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:05:43.227548 26487 heartbeater.cc:503] Master 127.5.122.62:39491 was elected leader, sending a full tablet report...
I20260807 03:05:43.227653 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.215667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.5.122.62:39491)
0807 03:05:43.215807 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.215813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.216240 (+ 427us) server_negotiation.cc:407] Connection header received
0807 03:05:43.216399 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.216404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.216469 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.216547 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:43.217891 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.218686 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.220007 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.220413 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.220520 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.222888 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:43.222893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:43.222896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:43.222926 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:43.225048 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.225698 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.225704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.225708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.225838 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.226176 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.226179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.226181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.226518 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:43.226755 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.227120 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.227337 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 03:05:43.231463 26198 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:05:43.238197 26198 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:05:43.261880 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.249185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36627 (local address 127.0.0.1:42352)
0807 03:05:43.249849 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.249894 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:43.250063 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:43.251254 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:43.251261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:43.251293 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:43.251675 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.251689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.254807 (+ 3118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.254813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.258657 (+ 3844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.258674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.260856 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.260862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.261078 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.261136 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 03:05:43.261246 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 03:05:43.261454 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":721,"threads_started":1}
I20260807 03:05:43.262887 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.250368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38274 (local address 127.5.122.1:44881)
0807 03:05:43.251356 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.251365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.253793 (+ 2428us) server_negotiation.cc:407] Connection header received
0807 03:05:43.253914 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.253923 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.254147 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.254362 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:43.255475 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.257230 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.259958 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.261384 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.261557 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.261791 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.262421 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.262574 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":697,"threads_started":1}
I20260807 03:05:43.263355 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.249717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:38274)
0807 03:05:43.253512 (+ 3795us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.253531 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:43.253668 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:43.254651 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:43.254657 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:43.254683 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:43.255189 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.255200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.257399 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.257405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.259749 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.259765 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.261948 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.261955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.262101 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.262129 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 03:05:43.262201 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 03:05:43.262307 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3667,"threads_started":1}
I20260807 03:05:43.263475 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.249616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.5.122.3:36627)
0807 03:05:43.250307 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.250339 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.250372 (+ 33us) server_negotiation.cc:407] Connection header received
0807 03:05:43.250470 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.250477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.250715 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.250929 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:43.251875 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.254617 (+ 2742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.258885 (+ 4268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.260612 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.262671 (+ 2059us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.262895 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.262964 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.263122 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":1660,"threads_started":1}
I20260807 03:05:43.267416 26804 tablet_service.cc:1467] Processing CreateTablet for tablet 1d965cd2e0fa423a8899dce5baf35217 (DEFAULT_TABLE table=test-table [id=5fd02665fce945ef9188aef6b96b0aec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:43.267633 26384 tablet_service.cc:1467] Processing CreateTablet for tablet 1d965cd2e0fa423a8899dce5baf35217 (DEFAULT_TABLE table=test-table [id=5fd02665fce945ef9188aef6b96b0aec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:43.268767 26384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d965cd2e0fa423a8899dce5baf35217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:43.268829 26804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d965cd2e0fa423a8899dce5baf35217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:43.270246 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.250662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39527 (local address 127.0.0.1:37194)
0807 03:05:43.262388 (+ 11726us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.262406 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:43.262512 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:43.263375 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:43.263379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:43.263654 (+ 275us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:43.263832 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.263843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.266070 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.266075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.267791 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.267806 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.269531 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.269538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.269734 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.269766 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:05:43.269861 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 03:05:43.269991 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11611,"thread_start_us":2593,"threads_started":1}
I20260807 03:05:43.270433 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.251120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.5.122.2:39527)
0807 03:05:43.260552 (+ 9432us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.260560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.262481 (+ 1921us) server_negotiation.cc:407] Connection header received
0807 03:05:43.262655 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.262663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.262891 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.264013 (+ 1122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:43.264334 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.265886 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.268007 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.269366 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.269493 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.269662 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.269928 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.270090 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9276,"thread_start_us":9153,"threads_started":1}
I20260807 03:05:43.274148 26599 tablet_service.cc:1467] Processing CreateTablet for tablet 1d965cd2e0fa423a8899dce5baf35217 (DEFAULT_TABLE table=test-table [id=5fd02665fce945ef9188aef6b96b0aec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:43.275490 26599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d965cd2e0fa423a8899dce5baf35217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:43.279876 26923 tablet_bootstrap.cc:492] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Bootstrap starting.
I20260807 03:05:43.282500 26925 tablet_bootstrap.cc:492] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Bootstrap starting.
I20260807 03:05:43.284420 26923 tablet_bootstrap.cc:654] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:43.285645 26923 log.cc:824] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:43.285693 26925 tablet_bootstrap.cc:654] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:43.286793 26925 log.cc:824] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:43.287747 26928 tablet_bootstrap.cc:492] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Bootstrap starting.
I20260807 03:05:43.288826 26923 tablet_bootstrap.cc:492] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: No bootstrap required, opened a new log
I20260807 03:05:43.289057 26923 ts_tablet_manager.cc:1397] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 03:05:43.290877 26928 tablet_bootstrap.cc:654] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:43.290993 26925 tablet_bootstrap.cc:492] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: No bootstrap required, opened a new log
I20260807 03:05:43.291234 26925 ts_tablet_manager.cc:1397] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 03:05:43.291810 26928 log.cc:824] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:43.293908 26928 tablet_bootstrap.cc:492] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: No bootstrap required, opened a new log
I20260807 03:05:43.294126 26928 ts_tablet_manager.cc:1397] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 03:05:43.299332 26923 raft_consensus.cc:348] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.299757 26923 raft_consensus.cc:374] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:43.299851 26923 raft_consensus.cc:729] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134a1de9819747159579124934a0a554, State: Initialized, Role: FOLLOWER
I20260807 03:05:43.300438 26923 consensus_queue.cc:260] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.300643 26928 raft_consensus.cc:348] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.301033 26928 raft_consensus.cc:374] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:43.301138 26928 raft_consensus.cc:729] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b748f970ed9f494daf27b1cabcafcdc3, State: Initialized, Role: FOLLOWER
I20260807 03:05:43.301075 26925 raft_consensus.cc:348] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.301486 26925 raft_consensus.cc:374] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:43.301594 26925 raft_consensus.cc:729] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749cb52b4c0d40b2bacd3e647cc227b9, State: Initialized, Role: FOLLOWER
I20260807 03:05:43.302253 26925 consensus_queue.cc:260] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.301748 26928 consensus_queue.cc:260] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.302762 26923 ts_tablet_manager.cc:1428] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260807 03:05:43.303833 26701 heartbeater.cc:503] Master 127.5.122.62:39491 was elected leader, sending a full tablet report...
I20260807 03:05:43.304379 26928 ts_tablet_manager.cc:1428] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260807 03:05:43.305543 26898 heartbeater.cc:503] Master 127.5.122.62:39491 was elected leader, sending a full tablet report...
I20260807 03:05:43.305752 26925 ts_tablet_manager.cc:1428] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 03:05:43.349316 26932 raft_consensus.cc:482] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:43.349570 26932 raft_consensus.cc:504] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.351133 26932 leader_election.cc:288] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627), b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527)
I20260807 03:05:43.358281 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.351410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:53659 (local address 127.5.122.3:36627)
0807 03:05:43.351572 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.351576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.351969 (+ 393us) server_negotiation.cc:407] Connection header received
0807 03:05:43.352191 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.352196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.352273 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.352382 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:43.353125 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.354115 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.356635 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.357395 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.357508 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.357610 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.357887 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.358033 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 03:05:43.358423 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.351342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36627 (local address 127.5.122.1:53659)
0807 03:05:43.351887 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.351904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:43.352044 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:43.352562 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:43.352567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:43.352599 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:43.352988 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.353000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.354242 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.354245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.356457 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.356474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.357595 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.357602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.357738 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.357761 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:05:43.357850 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 03:05:43.357973 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":20,"thread_start_us":253,"threads_started":1}
I20260807 03:05:43.358600 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.352229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:56429 (local address 127.5.122.2:39527)
0807 03:05:43.352446 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.352452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.352848 (+ 396us) server_negotiation.cc:407] Connection header received
0807 03:05:43.353024 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.353029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.353094 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.353182 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:43.353634 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.354538 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.356921 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.357688 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.357808 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.357897 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.358209 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.358355 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:05:43.358736 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.352042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39527 (local address 127.5.122.1:56429)
0807 03:05:43.352754 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.352775 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:43.352907 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:43.353204 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:43.353207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:43.353227 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:43.353501 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.353508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.354659 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.354664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.356782 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:43.356792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.357911 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.357918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:43.358062 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.358085 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:05:43.358166 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 03:05:43.358286 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":59,"thread_start_us":387,"threads_started":1}
I20260807 03:05:43.359578 26838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d965cd2e0fa423a8899dce5baf35217" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" is_pre_election: true
I20260807 03:05:43.359591 26631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d965cd2e0fa423a8899dce5baf35217" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748f970ed9f494daf27b1cabcafcdc3" is_pre_election: true
I20260807 03:05:43.360136 26838 raft_consensus.cc:2454] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 0.
I20260807 03:05:43.360141 26631 raft_consensus.cc:2454] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 0.
I20260807 03:05:43.360785 26316 leader_election.cc:302] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 134a1de9819747159579124934a0a554, b748f970ed9f494daf27b1cabcafcdc3; no voters:
I20260807 03:05:43.361105 26932 raft_consensus.cc:2781] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:05:43.361243 26932 raft_consensus.cc:482] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:05:43.361330 26932 raft_consensus.cc:3037] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:43.363494 26932 raft_consensus.cc:504] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.364010 26932 leader_election.cc:288] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 election: Requested vote from peers 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627), b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527)
I20260807 03:05:43.364393 26838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d965cd2e0fa423a8899dce5baf35217" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749cb52b4c0d40b2bacd3e647cc227b9"
I20260807 03:05:43.364480 26631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d965cd2e0fa423a8899dce5baf35217" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748f970ed9f494daf27b1cabcafcdc3"
I20260807 03:05:43.364553 26838 raft_consensus.cc:3037] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:43.364667 26631 raft_consensus.cc:3037] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:43.366683 26838 raft_consensus.cc:2454] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 1.
I20260807 03:05:43.366930 26631 raft_consensus.cc:2454] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 1.
I20260807 03:05:43.367009 26316 leader_election.cc:302] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 134a1de9819747159579124934a0a554, 749cb52b4c0d40b2bacd3e647cc227b9; no voters:
I20260807 03:05:43.367267 26932 raft_consensus.cc:2781] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:05:43.367664 26932 raft_consensus.cc:686] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 1 LEADER]: Becoming Leader. State: Replica: 134a1de9819747159579124934a0a554, State: Running, Role: LEADER
I20260807 03:05:43.368141 26932 consensus_queue.cc:237] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } }
I20260807 03:05:43.372382 26197 catalog_manager.cc:5495] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 reported cstate change: term changed from 0 to 1, leader changed from <none> to 134a1de9819747159579124934a0a554 (127.5.122.1). New cstate: current_term: 1 leader_uuid: "134a1de9819747159579124934a0a554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260807 03:05:43.412740 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.407958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.5.122.1:44881)
0807 03:05:43.408115 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.408120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.408136 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:05:43.408275 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.408279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.408363 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.408496 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:05:43.408862 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.409531 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.410893 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.411289 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.411378 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.411772 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:05:43.412254 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:05:43.412375 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.412457 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.412515 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 03:05:43.420043 26838 raft_consensus.cc:1261] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Refusing update from remote peer 134a1de9819747159579124934a0a554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:43.420078 26631 raft_consensus.cc:1261] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Refusing update from remote peer 134a1de9819747159579124934a0a554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:43.420881 26939 consensus_queue.cc:1035] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:05:43.421227 26932 consensus_queue.cc:1035] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 03:05:43.433934 26899 tablet.cc:2369] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:05:43.452521 26197 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:05:43.452790 26197 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:05:43.456399 26197 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Sending DeleteTablet for 3 replicas of tablet 1d965cd2e0fa423a8899dce5baf35217
I20260807 03:05:43.457428 26804 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d965cd2e0fa423a8899dce5baf35217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:43 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:42352
I20260807 03:05:43.457511 26599 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d965cd2e0fa423a8899dce5baf35217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:43 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:37194
I20260807 03:05:43.457608 26384 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d965cd2e0fa423a8899dce5baf35217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:43 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:38274
I20260807 03:05:43.458223 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 26292
I20260807 03:05:43.475391 26977 tablet_replica.cc:330] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: stopping tablet replica
I20260807 03:05:43.475782 26977 raft_consensus.cc:2229] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:05:43.476349 26977 raft_consensus.cc:2258] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:43.478379 26977 ts_tablet_manager.cc:1905] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 03:05:43 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:05:43.482815 26976 tablet_replica.cc:330] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: stopping tablet replica
I20260807 03:05:43.483211 26976 raft_consensus.cc:2229] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:05:43.483682 26976 raft_consensus.cc:2258] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:43.485790 26976 ts_tablet_manager.cc:1905] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:43.488185 26974 tablet_replica.cc:330] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: stopping tablet replica
I20260807 03:05:43.488588 26974 raft_consensus.cc:2229] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:05:43.489061 26974 raft_consensus.cc:2258] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:43.489912 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.459527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.5.122.1:44881)
0807 03:05:43.459702 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.459709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.470577 (+ 10868us) server_negotiation.cc:407] Connection header received
0807 03:05:43.470847 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.470853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.470937 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.471010 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:43.473306 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.474559 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.476487 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.477138 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.477233 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.482536 (+ 5303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:43.482550 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:43.482567 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:43.482616 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:43.486670 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.487617 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.487626 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.487630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.487801 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.488213 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.488218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.488222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.488563 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:43.488828 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.489338 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.489570 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 03:05:43.490613 26476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:43.492427 26974 ts_tablet_manager.cc:1905] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:43.493791 26976 ts_tablet_manager.cc:1918] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:43.493943 26976 log.cc:1196] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/1d965cd2e0fa423a8899dce5baf35217
I20260807 03:05:43.494477 26976 ts_tablet_manager.cc:1939] T 1d965cd2e0fa423a8899dce5baf35217 P 749cb52b4c0d40b2bacd3e647cc227b9: Deleting consensus metadata
I20260807 03:05:43.496310 26181 catalog_manager.cc:4841] TS 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627): tablet 1d965cd2e0fa423a8899dce5baf35217 (table test-table [id=5fd02665fce945ef9188aef6b96b0aec]) successfully deleted
I20260807 03:05:43.497960 26974 ts_tablet_manager.cc:1918] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:43.498116 26974 log.cc:1196] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/1d965cd2e0fa423a8899dce5baf35217
I20260807 03:05:43.498605 26974 ts_tablet_manager.cc:1939] T 1d965cd2e0fa423a8899dce5baf35217 P b748f970ed9f494daf27b1cabcafcdc3: Deleting consensus metadata
I20260807 03:05:43.500195 26181 catalog_manager.cc:4841] TS b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527): tablet 1d965cd2e0fa423a8899dce5baf35217 (table test-table [id=5fd02665fce945ef9188aef6b96b0aec]) successfully deleted
I20260807 03:05:43.593597 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 26292
W20260807 03:05:43.615077 26182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.1:44881 (error 108)
I20260807 03:05:43.616241 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 26498
I20260807 03:05:43.617100 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.616605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56146)
0807 03:05:43.616779 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.616900 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 03:05:43.617336 26182 catalog_manager.cc:4584] TS 134a1de9819747159579124934a0a554 (127.5.122.1:44881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d965cd2e0fa423a8899dce5baf35217: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Aug 07 03:05:43 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:05:43.631757 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.616984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.5.122.2:39527)
0807 03:05:43.617153 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.617160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.617334 (+ 174us) server_negotiation.cc:407] Connection header received
0807 03:05:43.617596 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.617602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.617694 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.617790 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:43.619848 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.620579 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.622021 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.622564 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.622654 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.626752 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:43.626767 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:43.626781 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:43.626820 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:43.629517 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.630099 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.630106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.630108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.630228 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.630540 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.630543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.630545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.630920 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:43.631139 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.631344 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.631499 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:05:43.632452 26692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:43.670902 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.670377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56152)
0807 03:05:43.670602 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.670714 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 03:05:43.672003 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.671578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56162)
0807 03:05:43.671730 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.671814 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 03:05:43.721496 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 26498
I20260807 03:05:43.732448 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.731972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56168)
0807 03:05:43.732157 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.732250 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 03:05:43.733388 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.733046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56180)
0807 03:05:43.733168 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.733245 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 03:05:43.741530 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 26712
Aug 07 03:05:43 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:05:43.756168 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.742189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.5.122.3:36627)
0807 03:05:43.742411 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:43.742417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:43.742436 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:05:43.742586 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:43.742592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:43.742663 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:43.742759 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:43.744075 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.744787 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.746123 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:43.746594 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:43.746707 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:43.750934 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:43.750956 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:43.750972 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:43.751005 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:43.753403 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.754051 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.754058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.754060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.754200 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:43.754673 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:43.754678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:43.754680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:43.755028 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:43.755323 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:43.755632 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 03:05:43.755854 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:05:43.756809 26892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:43.813781 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.813352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56182)
0807 03:05:43.813556 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.813643 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 03:05:43.814782 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:43.814403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56184)
0807 03:05:43.814592 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:43.814669 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 03:05:43.843335 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 26712
I20260807 03:05:43.865154 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 26150
I20260807 03:05:43.866240 26251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:43.973461 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 26150
I20260807 03:05:43.996668 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39491
--webserver_interface=127.5.122.62
--webserver_port=36755
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:44.312578 26988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:44.312832 26988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:44.312925 26988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:44.319765 26988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:44.319854 26988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:44.319905 26988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:44.319960 26988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:44.320003 26988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:44.329098 26988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=36755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:44.330148 26988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:44.331342 26988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:44.353408 26996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:44.353420 26999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:44.353602 26997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:44.354223 26988 server_base.cc:1034] running on GCE node
I20260807 03:05:44.355024 26988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:44.356388 26988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:44.357597 26988 hybrid_clock.cc:648] HybridClock initialized: now 1786071944357546 us; error 38 us; skew 500 ppm
I20260807 03:05:44.357973 26988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:44 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071944, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:44.361277 26988 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:05:44.362893 26988 webserver.cc:460] Webserver started at http://127.5.122.62:36755/ using document root <none> and password file <none>
I20260807 03:05:44.363588 26988 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:44.363726 26988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:44.369330 26988 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 03:05:44.372121 27008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:44.373381 26988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 03:05:44.373546 26988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29"
format_stamp: "Formatted at 2026-08-07 03:05:41 on dist-test-slave-fgb3"
server_key: "60da7541437f834baa0827496c426e80"
server_key_iv: "503ac9110cb40d1176bcc96fdd7f6770"
server_key_version: "encryptionkey@0"
I20260807 03:05:44.374641 26988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:44.401340 26988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:44.405718 26988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:44.406013 26988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:44.425153 26988 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:39491
I20260807 03:05:44.425173 27083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:39491 every 8 connection(s)
I20260807 03:05:44.426635 26988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:05:44.429730 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 26988
I20260807 03:05:44.430722 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:44881
--local_ip_for_outbound_sockets=127.5.122.1
--tserver_master_addrs=127.5.122.62:39491
--webserver_port=38383
--webserver_interface=127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:05:44.434998 27084 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:05:44.439309 27084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Bootstrap starting.
I20260807 03:05:44.459087 27084 log.cc:824] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:44.471275 27084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:05:44.471765 27084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Bootstrap complete.
I20260807 03:05:44.481261 27084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:44.482123 27084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0d7bd5b2ba4d00ab6615b6ed9caa29, State: Initialized, Role: FOLLOWER
I20260807 03:05:44.482748 27084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:44.482970 27084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:05:44.483081 27084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:05:44.483239 27084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 1 FOLLOWER]: Advancing to term 2
I20260807 03:05:44.486943 27084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:44.487416 27084 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0d7bd5b2ba4d00ab6615b6ed9caa29; no voters:
I20260807 03:05:44.487975 27084 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 03:05:44.488116 27093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 03:05:44.489220 27093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 LEADER]: Becoming Leader. State: Replica: 1b0d7bd5b2ba4d00ab6615b6ed9caa29, State: Running, Role: LEADER
I20260807 03:05:44.489780 27093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:44.490222 27084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:44.492157 27101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0d7bd5b2ba4d00ab6615b6ed9caa29. Latest consensus state: current_term: 2 leader_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } } }
I20260807 03:05:44.492524 27101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:44.493037 27098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } } }
I20260807 03:05:44.493320 27098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:44.497834 27107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:05:44.506340 27107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:05:44.507809 27107 catalog_manager.cc:1259] Loaded cluster ID: cca53674857548b7b8b847b5e28a8b23
I20260807 03:05:44.507908 27107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:05:44.511195 27107 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 03:05:44.530027 27107 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 03:05:44.536355 27123 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 03:05:44.538230 27107 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:05:44.539454 27107 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Loaded TSK: 0
I20260807 03:05:44.540482 27107 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 03:05:44.788866 27086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:44.789145 27086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:44.789268 27086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:44.796700 27086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:44.796813 27086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:44.796964 27086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:05:44.806851 27086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:44881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=38383
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:44.808108 27086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:44.809441 27086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:44.823959 27129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:44.824093 27136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:44.823925 27134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:44.825433 27086 server_base.cc:1034] running on GCE node
I20260807 03:05:44.826028 27086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:44.827080 27086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:44.828295 27086 hybrid_clock.cc:648] HybridClock initialized: now 1786071944828247 us; error 44 us; skew 500 ppm
I20260807 03:05:44.828606 27086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:44 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071944, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:44.836993 27086 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:05:44.838575 27086 webserver.cc:460] Webserver started at http://127.5.122.1:38383/ using document root <none> and password file <none>
I20260807 03:05:44.839257 27086 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:44.839382 27086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:44.844725 27086 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 03:05:44.847620 27147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:44.848942 27086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:05:44.849085 27086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "134a1de9819747159579124934a0a554"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "216fdeb6c4c828327f54d7277475c15b"
server_key_iv: "9c27b1c09dd83b704ce3e31f0158de99"
server_key_version: "encryptionkey@0"
I20260807 03:05:44.850021 27086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:44.863757 27086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:44.867797 27086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:44.868058 27086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:44.869031 27086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:44.870965 27158 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 03:05:44.878003 27158 ts_tablet_manager.cc:1852] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 03:05:44.879503 27158 ts_tablet_manager.cc:1905] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:44.887161 27158 ts_tablet_manager.cc:1918] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:44.887324 27158 log.cc:1196] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/1d965cd2e0fa423a8899dce5baf35217
I20260807 03:05:44.887733 27158 ts_tablet_manager.cc:1939] T 1d965cd2e0fa423a8899dce5baf35217 P 134a1de9819747159579124934a0a554: Deleting consensus metadata
I20260807 03:05:44.888060 27086 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 03:05:44.888178 27086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.000s sys 0.000s
I20260807 03:05:44.888295 27086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:44.888365 27086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:44.917811 27086 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:44881
I20260807 03:05:44.917878 27311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:44881 every 8 connection(s)
I20260807 03:05:44.919101 27086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:05:44.919531 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 27086
I20260807 03:05:44.920303 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:39527
--local_ip_for_outbound_sockets=127.5.122.2
--tserver_master_addrs=127.5.122.62:39491
--webserver_port=34873
--webserver_interface=127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:05:44 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071944, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:44.942677 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:44.922137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:34063 (local address 127.5.122.62:39491)
0807 03:05:44.922810 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:44.922825 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:44.923550 (+ 725us) server_negotiation.cc:407] Connection header received
0807 03:05:44.925807 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:44.925819 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:44.926117 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:44.926438 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:44.928233 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:44.929415 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:44.931463 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:44.932086 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:44.932268 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:44.935957 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:44.935965 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:44.935977 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:44.936014 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:44.938703 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:44.939464 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:44.939470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:44.939474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:44.939626 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:44.940055 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:44.940057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:44.940059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:44.940470 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:44.940611 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:44.941464 (+ 853us) server_negotiation.cc:299] Negotiation successful
0807 03:05:44.941747 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":278,"threads_started":1}
I20260807 03:05:44.942909 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:44.922242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:34063)
0807 03:05:44.923132 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:44.923185 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:44.925325 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:44.926620 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:44.926629 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:44.927337 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:44.928030 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:44.928054 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:44.929566 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:44.929571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:44.931328 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:44.931342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:44.932311 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:44.932319 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:44.932565 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:44.933150 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:44.933177 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:44.935695 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:44.938831 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:44.938839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:44.938852 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:44.939356 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:44.939742 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:44.939746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:44.939749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:44.939965 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:44.940596 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:44.940603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:44.941015 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 03:05:44.941294 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 03:05:44.941734 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":375,"threads_started":1}
I20260807 03:05:44.945492 27312 heartbeater.cc:347] Connected to a master server at 127.5.122.62:39491
I20260807 03:05:44.945760 27312 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:44.946421 27312 heartbeater.cc:511] Master 127.5.122.62:39491 requested a full tablet report, sending...
I20260807 03:05:44.948380 27029 ts_manager.cc:194] Registered new tserver with Master: 134a1de9819747159579124934a0a554 (127.5.122.1:44881)
I20260807 03:05:44.949817 27029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:34063
W20260807 03:05:45.244055 27319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:45.244307 27319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:45.244455 27319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:45.251443 27319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:45.251547 27319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:45.251704 27319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:05:45.261448 27319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:39527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=34873
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:45.262589 27319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:45.263813 27319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:45.275940 27326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:45.276027 27327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:45.276176 27329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:45.277096 27319 server_base.cc:1034] running on GCE node
I20260807 03:05:45.277683 27319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:45.278681 27319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:45.279871 27319 hybrid_clock.cc:648] HybridClock initialized: now 1786071945279824 us; error 42 us; skew 500 ppm
I20260807 03:05:45.280171 27319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:45 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:45.283146 27319 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:05:45.284399 27319 webserver.cc:460] Webserver started at http://127.5.122.2:34873/ using document root <none> and password file <none>
I20260807 03:05:45.285022 27319 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:45.285127 27319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:45.289984 27319 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:05:45.292557 27343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:45.293761 27319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:05:45.293902 27319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "b748f970ed9f494daf27b1cabcafcdc3"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "d95113664e19273dc25b2e299c476303"
server_key_iv: "1cddd782e4818fe9a33a864300e52259"
server_key_version: "encryptionkey@0"
I20260807 03:05:45.294896 27319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:45.322190 27319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:45.326220 27319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:45.326493 27319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:45.327455 27319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:45.328729 27319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:45.328823 27319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:45.328924 27319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:45.329005 27319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:45.358474 27319 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:39527
I20260807 03:05:45.358553 27509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:39527 every 8 connection(s)
I20260807 03:05:45.359766 27319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:05:45.369079 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 27319
I20260807 03:05:45.369653 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:36627
--local_ip_for_outbound_sockets=127.5.122.3
--tserver_master_addrs=127.5.122.62:39491
--webserver_port=41043
--webserver_interface=127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:05:45 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:45.377943 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.361997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:55727 (local address 127.5.122.62:39491)
0807 03:05:45.362180 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:45.362185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:45.363340 (+ 1155us) server_negotiation.cc:407] Connection header received
0807 03:05:45.365188 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:45.365194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:45.365269 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:45.365367 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:45.366776 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.367450 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.368963 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.369415 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.369524 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.372751 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:45.372756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:45.372760 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:45.372792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:45.375111 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:45.375809 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:45.375813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:45.375816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:45.375950 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:45.376351 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:45.376354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:45.376356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:45.376607 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:45.376740 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:45.377384 (+ 644us) server_negotiation.cc:299] Negotiation successful
0807 03:05:45.377617 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 03:05:45.378942 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.362472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:55727)
0807 03:05:45.363155 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:45.363197 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:45.364922 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:45.365597 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:45.365605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:45.366070 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:45.366605 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.366626 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.367603 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.367607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.368806 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.368818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.369623 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.369631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.369907 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.370521 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:45.370542 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:45.372562 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:45.375237 (+ 2675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:45.375245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:45.375259 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:45.375690 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:45.376053 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:45.376058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:45.376062 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:45.376233 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:45.376737 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:45.376745 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:45.377026 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 03:05:45.377237 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 03:05:45.377691 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":288,"threads_started":1}
I20260807 03:05:45.380252 27510 heartbeater.cc:347] Connected to a master server at 127.5.122.62:39491
I20260807 03:05:45.380553 27510 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:45.381373 27510 heartbeater.cc:511] Master 127.5.122.62:39491 requested a full tablet report, sending...
I20260807 03:05:45.382764 27029 ts_manager.cc:194] Registered new tserver with Master: b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527)
I20260807 03:05:45.383791 27029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:55727
W20260807 03:05:45.692378 27519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:45.692641 27519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:45.692739 27519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:45.699937 27519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:45.700043 27519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:45.700157 27519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:05:45.710410 27519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:36627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=41043
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:45.711647 27519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:45.712953 27519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:45.724920 27526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:45.725082 27534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:45.725049 27530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:45.725950 27519 server_base.cc:1034] running on GCE node
I20260807 03:05:45.726526 27519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:45.727460 27519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:45.728654 27519 hybrid_clock.cc:648] HybridClock initialized: now 1786071945728604 us; error 48 us; skew 500 ppm
I20260807 03:05:45.728940 27519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:45 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:45.731875 27519 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:05:45.733214 27519 webserver.cc:460] Webserver started at http://127.5.122.3:41043/ using document root <none> and password file <none>
I20260807 03:05:45.733852 27519 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:45.733974 27519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:45.738915 27519 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:05:45.741350 27542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:45.742619 27519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:05:45.742765 27519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "749cb52b4c0d40b2bacd3e647cc227b9"
format_stamp: "Formatted at 2026-08-07 03:05:43 on dist-test-slave-fgb3"
server_key: "85f75fd20c436ddaebb50393ffc19ebf"
server_key_iv: "5aa0a4fef66ae5061d2ebdb3022bf787"
server_key_version: "encryptionkey@0"
I20260807 03:05:45.743681 27519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:45.757463 27519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:45.761197 27519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:45.761426 27519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:45.762279 27519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:45.763509 27519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:45.763600 27519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:45.763700 27519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:45.763770 27519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:45.795367 27519 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36627
I20260807 03:05:45.795433 27709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36627 every 8 connection(s)
I20260807 03:05:45.796770 27519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:05:45.797281 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 27519
Aug 07 03:05:45 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:05:45.813884 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.798497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.5.122.62:39491)
0807 03:05:45.798703 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:45.798710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:45.799486 (+ 776us) server_negotiation.cc:407] Connection header received
0807 03:05:45.799572 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:45.799579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:45.799692 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:45.799835 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:45.802019 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.802868 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.804220 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.804650 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.804768 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.807947 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:45.807952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:45.807955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:45.807989 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:45.810638 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:45.811494 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:45.811498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:45.811501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:45.811629 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:45.812033 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:45.812037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:45.812040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:45.812450 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:45.812780 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:45.813250 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 03:05:45.813486 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:05:45.818349 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.799400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:60295 (local address 127.5.122.62:39491)
0807 03:05:45.799876 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:45.799882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:45.801012 (+ 1130us) server_negotiation.cc:407] Connection header received
0807 03:05:45.803209 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:45.803215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:45.803295 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:45.803395 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:45.804909 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.805679 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.807871 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.808351 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.808459 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.812126 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:45.812132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:45.812136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:45.812175 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:45.815554 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:45.816420 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:45.816426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:45.816430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:45.816546 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:45.816914 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:45.816916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:45.816918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:45.817174 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:45.817278 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:45.817884 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 03:05:45.818040 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":221,"threads_started":1}
I20260807 03:05:45.819352 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.800012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:60295)
0807 03:05:45.800861 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:45.800906 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:45.802946 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:45.803652 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:45.803660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:45.804150 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:45.804717 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.804735 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.805865 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.805869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.807716 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.807728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.808570 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.808578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.808794 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.809394 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:45.809418 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:45.811908 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:45.815678 (+ 3770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:45.815687 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:45.815702 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:45.816297 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:45.816635 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:45.816638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:45.816640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:45.816818 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:45.817283 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:45.817289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:45.817605 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 03:05:45.817816 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 03:05:45.818209 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":363,"threads_started":1}
I20260807 03:05:45.820598 27712 heartbeater.cc:347] Connected to a master server at 127.5.122.62:39491
I20260807 03:05:45.820884 27712 heartbeater.cc:464] Registering TS with master...
I20260807 03:05:45.821568 27712 heartbeater.cc:511] Master 127.5.122.62:39491 requested a full tablet report, sending...
I20260807 03:05:45.822907 27029 ts_manager.cc:194] Registered new tserver with Master: 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
I20260807 03:05:45.823735 27029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:60295
I20260807 03:05:45.828554 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:05:45.834672 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.829484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.5.122.62:39491)
0807 03:05:45.829635 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:45.829639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:45.830069 (+ 430us) server_negotiation.cc:407] Connection header received
0807 03:05:45.830237 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:45.830241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:45.830457 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:45.830567 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:05:45.830931 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.831605 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.832911 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.833301 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.833400 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.833799 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:05:45.834233 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:05:45.834393 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:45.834431 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 03:05:45.834493 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:05:45.835544 27029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:05:45.842619 27029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:05:45.875937 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.853992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.5.122.2:39527)
0807 03:05:45.866042 (+ 12050us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:45.866050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:45.867028 (+ 978us) server_negotiation.cc:407] Connection header received
0807 03:05:45.867135 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:45.867143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:45.867416 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:45.867577 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:45.868669 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.870238 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.873226 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.874777 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.874907 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.875160 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:45.875407 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 03:05:45.875585 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11886,"thread_start_us":11765,"threads_started":1}
I20260807 03:05:45.876029 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.853680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39527 (local address 127.0.0.1:44372)
0807 03:05:45.866550 (+ 12870us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:45.866611 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:45.866820 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:45.867856 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:45.867864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:45.867904 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:45.868438 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.868451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.870443 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.870449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.873066 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.873078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.874971 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.874978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.875163 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.875211 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 03:05:45.875347 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 03:05:45.875567 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12551,"mutex_wait_us":142,"spinlock_wait_cycles":2304,"thread_start_us":12832,"threads_started":1}
I20260807 03:05:45.877525 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.853994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56190)
0807 03:05:45.866772 (+ 12778us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:45.866787 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:45.866995 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:45.867856 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:45.867864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:45.867916 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:45.868437 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.868451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.870443 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.870448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.874458 (+ 4010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.874473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.876596 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.876604 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.876830 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.876877 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 03:05:45.876974 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 03:05:45.877099 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12664,"mutex_wait_us":131,"spinlock_wait_cycles":2432,"thread_start_us":12121,"threads_started":1}
I20260807 03:05:45.877575 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.854193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.5.122.1:44881)
0807 03:05:45.865374 (+ 11181us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:45.865384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:45.866892 (+ 1508us) server_negotiation.cc:407] Connection header received
0807 03:05:45.867168 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:45.867174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:45.867401 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:45.867568 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:45.868678 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.870238 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.874693 (+ 4455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.876083 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.876205 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.876404 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:45.877042 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 03:05:45.877219 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10980,"thread_start_us":11198,"threads_started":1}
I20260807 03:05:45.879454 27216 tablet_service.cc:1467] Processing CreateTablet for tablet 6bafab2c296548f9bd1ab51c65824956 (DEFAULT_TABLE table=test-table [id=e37787cef5f845a98a104a1d18c47e7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:45.880389 27410 tablet_service.cc:1467] Processing CreateTablet for tablet 6bafab2c296548f9bd1ab51c65824956 (DEFAULT_TABLE table=test-table [id=e37787cef5f845a98a104a1d18c47e7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:45.880918 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bafab2c296548f9bd1ab51c65824956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:45.881472 27410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bafab2c296548f9bd1ab51c65824956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:45.882268 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.867959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36627 (local address 127.0.0.1:52850)
0807 03:05:45.872319 (+ 4360us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:45.872341 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:45.872447 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:45.873155 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:45.873160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:45.873186 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:45.873633 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.873645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.875509 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.875514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.879023 (+ 3509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:45.879039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.881452 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.881459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:45.881637 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.881670 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:05:45.881758 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 03:05:45.881892 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4228,"mutex_wait_us":1,"thread_start_us":4038,"threads_started":1}
I20260807 03:05:45.883081 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:45.867429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.5.122.3:36627)
0807 03:05:45.872682 (+ 5253us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:45.872689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:45.872715 (+ 26us) server_negotiation.cc:407] Connection header received
0807 03:05:45.872792 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:45.872798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:45.872985 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:45.873138 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:45.873845 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.875354 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.879186 (+ 3832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:45.880941 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:45.881076 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:45.881292 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:45.882546 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0807 03:05:45.882719 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5114,"thread_start_us":4937,"threads_started":1}
I20260807 03:05:45.889704 27733 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Bootstrap starting.
I20260807 03:05:45.890189 27618 tablet_service.cc:1467] Processing CreateTablet for tablet 6bafab2c296548f9bd1ab51c65824956 (DEFAULT_TABLE table=test-table [id=e37787cef5f845a98a104a1d18c47e7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:05:45.892385 27618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bafab2c296548f9bd1ab51c65824956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:05:45.893090 27733 tablet_bootstrap.cc:654] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:45.894052 27733 log.cc:824] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:45.900445 27734 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Bootstrap starting.
I20260807 03:05:45.901887 27733 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: No bootstrap required, opened a new log
I20260807 03:05:45.902173 27733 ts_tablet_manager.cc:1397] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260807 03:05:45.904007 27734 tablet_bootstrap.cc:654] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:45.905220 27734 log.cc:824] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:45.905979 27739 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Bootstrap starting.
I20260807 03:05:45.908550 27734 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: No bootstrap required, opened a new log
I20260807 03:05:45.908828 27734 ts_tablet_manager.cc:1397] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 03:05:45.909514 27739 tablet_bootstrap.cc:654] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Neither blocks nor log segments found. Creating new log.
I20260807 03:05:45.910754 27739 log.cc:824] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:45.911147 27733 raft_consensus.cc:348] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:45.911509 27733 raft_consensus.cc:374] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:45.911602 27733 raft_consensus.cc:729] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134a1de9819747159579124934a0a554, State: Initialized, Role: FOLLOWER
I20260807 03:05:45.912153 27733 consensus_queue.cc:260] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:45.913163 27312 heartbeater.cc:503] Master 127.5.122.62:39491 was elected leader, sending a full tablet report...
I20260807 03:05:45.913763 27733 ts_tablet_manager.cc:1428] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 03:05:45.919685 27734 raft_consensus.cc:348] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:45.920081 27739 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: No bootstrap required, opened a new log
I20260807 03:05:45.920123 27734 raft_consensus.cc:374] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:45.920209 27734 raft_consensus.cc:729] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b748f970ed9f494daf27b1cabcafcdc3, State: Initialized, Role: FOLLOWER
I20260807 03:05:45.920363 27739 ts_tablet_manager.cc:1397] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260807 03:05:45.920881 27734 consensus_queue.cc:260] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
W20260807 03:05:45.921207 27314 tablet.cc:2369] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:05:45.927088 27510 heartbeater.cc:503] Master 127.5.122.62:39491 was elected leader, sending a full tablet report...
I20260807 03:05:45.927816 27734 ts_tablet_manager.cc:1428] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Time spent starting tablet: real 0.019s user 0.015s sys 0.004s
I20260807 03:05:45.930639 27739 raft_consensus.cc:348] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:45.931047 27739 raft_consensus.cc:374] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:05:45.931154 27739 raft_consensus.cc:729] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749cb52b4c0d40b2bacd3e647cc227b9, State: Initialized, Role: FOLLOWER
I20260807 03:05:45.931798 27739 consensus_queue.cc:260] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:45.932832 27712 heartbeater.cc:503] Master 127.5.122.62:39491 was elected leader, sending a full tablet report...
I20260807 03:05:45.933470 27739 ts_tablet_manager.cc:1428] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
W20260807 03:05:46.048544 27713 tablet.cc:2369] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:05:46.111594 27512 tablet.cc:2369] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:05:46.193768 27746 raft_consensus.cc:482] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:46.194083 27746 raft_consensus.cc:504] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:46.195777 27746 leader_election.cc:288] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527), 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
I20260807 03:05:46.203400 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.196252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:35463 (local address 127.5.122.2:39527)
0807 03:05:46.196442 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:46.196446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:46.196833 (+ 387us) server_negotiation.cc:407] Connection header received
0807 03:05:46.197065 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:46.197071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:46.197152 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:46.197254 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:46.197979 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.199024 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.201460 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.202588 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.202706 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.202813 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:46.203078 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 03:05:46.203211 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:05:46.203799 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.197090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:56537 (local address 127.5.122.3:36627)
0807 03:05:46.197340 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:46.197346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:46.197738 (+ 392us) server_negotiation.cc:407] Connection header received
0807 03:05:46.197933 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:46.197938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:46.198026 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:46.198120 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:05:46.198693 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.199550 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.201782 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.202946 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.203050 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.203144 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:46.203496 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 03:05:46.203622 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:05:46.203801 27750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.196093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39527 (local address 127.5.122.1:35463)
0807 03:05:46.196725 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.196749 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:46.196904 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:46.197440 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:46.197445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:46.197477 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:46.197827 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:46.197838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.199137 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.199140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:46.201279 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:46.201295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.202781 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.202785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:46.202923 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.202948 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 03:05:46.203030 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 03:05:46.203167 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":26,"thread_start_us":306,"threads_started":1}
I20260807 03:05:46.203908 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.196900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36627 (local address 127.5.122.1:56537)
0807 03:05:46.197637 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.197656 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:46.197802 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:46.198130 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:46.198133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:46.198152 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:05:46.198515 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:46.198526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.199660 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.199665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:46.201642 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:46.201653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.203140 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.203146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:46.203315 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.203347 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:05:46.203450 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 03:05:46.203568 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":1,"thread_start_us":384,"threads_started":1}
I20260807 03:05:46.204846 27440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafab2c296548f9bd1ab51c65824956" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748f970ed9f494daf27b1cabcafcdc3" is_pre_election: true
I20260807 03:05:46.204846 27651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafab2c296548f9bd1ab51c65824956" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" is_pre_election: true
I20260807 03:05:46.205315 27440 raft_consensus.cc:2454] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 0.
I20260807 03:05:46.205319 27651 raft_consensus.cc:2454] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 0.
I20260807 03:05:46.205971 27150 leader_election.cc:302] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 134a1de9819747159579124934a0a554, b748f970ed9f494daf27b1cabcafcdc3; no voters:
I20260807 03:05:46.206378 27746 raft_consensus.cc:2781] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:05:46.206544 27746 raft_consensus.cc:482] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:05:46.206646 27746 raft_consensus.cc:3037] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:46.208820 27746 raft_consensus.cc:504] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:46.209403 27746 leader_election.cc:288] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 election: Requested vote from peers b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527), 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
I20260807 03:05:46.209849 27440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafab2c296548f9bd1ab51c65824956" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748f970ed9f494daf27b1cabcafcdc3"
I20260807 03:05:46.209900 27651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bafab2c296548f9bd1ab51c65824956" candidate_uuid: "134a1de9819747159579124934a0a554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749cb52b4c0d40b2bacd3e647cc227b9"
I20260807 03:05:46.210037 27440 raft_consensus.cc:3037] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:46.210098 27651 raft_consensus.cc:3037] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:05:46.212257 27651 raft_consensus.cc:2454] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 1.
I20260807 03:05:46.212327 27440 raft_consensus.cc:2454] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 134a1de9819747159579124934a0a554 in term 1.
I20260807 03:05:46.212694 27150 leader_election.cc:302] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 134a1de9819747159579124934a0a554, 749cb52b4c0d40b2bacd3e647cc227b9; no voters:
I20260807 03:05:46.212987 27746 raft_consensus.cc:2781] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:05:46.213407 27746 raft_consensus.cc:686] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 LEADER]: Becoming Leader. State: Replica: 134a1de9819747159579124934a0a554, State: Running, Role: LEADER
I20260807 03:05:46.213907 27746 consensus_queue.cc:237] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:46.218299 27029 catalog_manager.cc:5495] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 reported cstate change: term changed from 0 to 1, leader changed from <none> to 134a1de9819747159579124934a0a554 (127.5.122.1). New cstate: current_term: 1 leader_uuid: "134a1de9819747159579124934a0a554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } health_report { overall_health: UNKNOWN } } }
I20260807 03:05:46.246297 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.240898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.5.122.1:44881)
0807 03:05:46.241124 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:46.241130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:46.241149 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:05:46.241327 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:46.241331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:46.241412 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:46.241551 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:05:46.241938 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.242796 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.244183 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.244574 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.244681 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.245202 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:05:46.245664 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:05:46.245798 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:46.245910 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 03:05:46.245994 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 03:05:46.254127 27440 raft_consensus.cc:1261] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Refusing update from remote peer 134a1de9819747159579124934a0a554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:46.254163 27651 raft_consensus.cc:1261] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Refusing update from remote peer 134a1de9819747159579124934a0a554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:05:46.255036 27757 consensus_queue.cc:1035] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:05:46.255417 27746 consensus_queue.cc:1035] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:05:46.280114 27028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:05:46.280400 27028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:05:46.283923 27028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Sending DeleteTablet for 3 replicas of tablet 6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:46.285127 27216 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bafab2c296548f9bd1ab51c65824956 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:46 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:56190
I20260807 03:05:46.285260 27410 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bafab2c296548f9bd1ab51c65824956 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:46 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:44372
I20260807 03:05:46.285409 27618 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bafab2c296548f9bd1ab51c65824956 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:05:46 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:52850
I20260807 03:05:46.286388 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 27086
I20260807 03:05:46.296368 27778 tablet_replica.cc:330] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: stopping tablet replica
I20260807 03:05:46.297011 27778 raft_consensus.cc:2229] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:05:46.297708 27778 raft_consensus.cc:2258] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:46.300879 27776 tablet_replica.cc:330] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: stopping tablet replica
I20260807 03:05:46.301044 27781 tablet_replica.cc:330] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: stopping tablet replica
I20260807 03:05:46.301353 27776 raft_consensus.cc:2229] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:05:46.301395 27781 raft_consensus.cc:2229] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:05:46.301863 27781 raft_consensus.cc:2258] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:46.302137 27776 raft_consensus.cc:2258] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:05:46.304163 27781 ts_tablet_manager.cc:1905] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:46.304337 27778 ts_tablet_manager.cc:1905] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:05:46.307293 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.287506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.5.122.1:44881)
0807 03:05:46.287678 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:46.287684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:46.287703 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:05:46.287765 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:46.287771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:46.287863 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:46.287975 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:46.289822 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.291123 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.293123 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.293862 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.293986 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.297608 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:46.297621 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:46.297637 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:46.297688 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:46.302277 (+ 4589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:46.303604 (+ 1327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:46.303613 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:46.303617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:46.303782 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:46.304247 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:46.304252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:46.304255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:46.304707 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:46.304997 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:46.306679 (+ 1682us) server_negotiation.cc:299] Negotiation successful
0807 03:05:46.306908 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:05:46.307904 27304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:46.311041 27781 ts_tablet_manager.cc:1918] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:46.311203 27781 log.cc:1196] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:46.311124 27778 ts_tablet_manager.cc:1918] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:05:46.311447 27778 log.cc:1196] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:46.311748 27781 ts_tablet_manager.cc:1939] T 6bafab2c296548f9bd1ab51c65824956 P 749cb52b4c0d40b2bacd3e647cc227b9: Deleting consensus metadata
I20260807 03:05:46.312004 27778 ts_tablet_manager.cc:1939] T 6bafab2c296548f9bd1ab51c65824956 P b748f970ed9f494daf27b1cabcafcdc3: Deleting consensus metadata
I20260807 03:05:46.313584 27010 catalog_manager.cc:4841] TS 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627): tablet 6bafab2c296548f9bd1ab51c65824956 (table test-table [id=e37787cef5f845a98a104a1d18c47e7e]) successfully deleted
I20260807 03:05:46.313868 27010 catalog_manager.cc:4841] TS b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527): tablet 6bafab2c296548f9bd1ab51c65824956 (table test-table [id=e37787cef5f845a98a104a1d18c47e7e]) successfully deleted
I20260807 03:05:46.416286 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 27086
I20260807 03:05:46.416350 27776 ts_tablet_manager.cc:1905] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 03:05:46.440447 27011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.1:44881 (error 108)
I20260807 03:05:46.440701 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 27319
I20260807 03:05:46.442436 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.441649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56208)
0807 03:05:46.442095 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.442222 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":237,"threads_started":1}
W20260807 03:05:46.442653 27011 catalog_manager.cc:4584] TS 134a1de9819747159579124934a0a554 (127.5.122.1:44881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6bafab2c296548f9bd1ab51c65824956: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
I20260807 03:05:46.453264 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.441494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.5.122.2:39527)
0807 03:05:46.441680 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:46.441685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:46.441848 (+ 163us) server_negotiation.cc:407] Connection header received
0807 03:05:46.442029 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:46.442034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:46.442122 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:46.442241 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:46.443690 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.444380 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.445504 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.445895 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.445981 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.448544 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:46.448557 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:46.448571 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:46.448602 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:46.451078 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:46.451637 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:46.451643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:46.451645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:46.451764 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:46.452036 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:46.452039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:46.452041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:46.452356 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:46.452576 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:46.452779 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 03:05:46.452932 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:05:46.453837 27504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:46.496294 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.495867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56214)
0807 03:05:46.496046 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.496144 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 03:05:46.497303 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.496931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56216)
0807 03:05:46.497096 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.497166 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 03:05:46.541329 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 27319
I20260807 03:05:46.557641 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.557213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56220)
0807 03:05:46.557404 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.557494 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 03:05:46.558648 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.558305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56228)
0807 03:05:46.558447 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.558518 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 03:05:46.562690 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 27519
I20260807 03:05:46.579334 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.567491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.5.122.3:36627)
0807 03:05:46.567672 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:46.567677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:46.567698 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:05:46.567841 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:46.567845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:46.567930 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:46.568012 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:46.569404 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.570098 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.571423 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:46.571805 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:46.571924 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:46.574243 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:46.574268 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:46.574284 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:46.574341 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:46.576646 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:46.577290 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:46.577298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:46.577300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:46.577446 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:46.577798 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:46.577803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:46.577806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:46.578169 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:46.578665 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:46.578822 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 03:05:46.579034 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:05:46.579977 27703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:46.639261 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.638795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56232)
0807 03:05:46.639008 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.639106 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 03:05:46.640282 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:46.639889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44881 (local address 127.0.0.1:56236)
0807 03:05:46.640078 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:46.640145 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:44881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 03:05:46.666736 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 27519
I20260807 03:05:46.688576 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 26988
I20260807 03:05:46.689723 27075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:05:46.805045 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 26988
I20260807 03:05:46.832259 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39491
--webserver_interface=127.5.122.62
--webserver_port=36755
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:05:47.150681 27791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:47.150930 27791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:47.151057 27791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:47.158051 27791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:05:47.158154 27791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:47.158227 27791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:47.158305 27791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:05:47.158382 27791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:05:47.167989 27791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:39491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=36755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:47.169277 27791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:47.170579 27791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:47.181393 27801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:47.181454 27803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:47.181489 27805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:47.182202 27791 server_base.cc:1034] running on GCE node
I20260807 03:05:47.182904 27791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:47.184154 27791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:47.185343 27791 hybrid_clock.cc:648] HybridClock initialized: now 1786071947185293 us; error 40 us; skew 500 ppm
I20260807 03:05:47.185654 27791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:47 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:47.188539 27791 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:05:47.189853 27791 webserver.cc:460] Webserver started at http://127.5.122.62:36755/ using document root <none> and password file <none>
I20260807 03:05:47.190514 27791 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:47.190629 27791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:47.195644 27791 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 03:05:47.198273 27813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:47.199563 27791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 03:05:47.199723 27791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29"
format_stamp: "Formatted at 2026-08-07 03:05:41 on dist-test-slave-fgb3"
server_key: "60da7541437f834baa0827496c426e80"
server_key_iv: "503ac9110cb40d1176bcc96fdd7f6770"
server_key_version: "encryptionkey@0"
I20260807 03:05:47.200668 27791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:47.223742 27791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:47.227893 27791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:47.228139 27791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:47.244721 27791 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:39491
I20260807 03:05:47.244735 27882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:39491 every 8 connection(s)
I20260807 03:05:47.246135 27791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:05:47.253041 27883 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:05:47.254812 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 27791
I20260807 03:05:47.255581 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:44881
--local_ip_for_outbound_sockets=127.5.122.1
--tserver_master_addrs=127.5.122.62:39491
--webserver_port=38383
--webserver_interface=127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:05:47.256215 27883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Bootstrap starting.
I20260807 03:05:47.275815 27883 log.cc:824] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:47.291586 27883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:05:47.292089 27883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29: Bootstrap complete.
I20260807 03:05:47.300809 27883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:47.301648 27883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0d7bd5b2ba4d00ab6615b6ed9caa29, State: Initialized, Role: FOLLOWER
I20260807 03:05:47.302263 27883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:47.302503 27883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:05:47.302619 27883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:05:47.302770 27883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 2 FOLLOWER]: Advancing to term 3
I20260807 03:05:47.306728 27883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:47.307192 27883 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0d7bd5b2ba4d00ab6615b6ed9caa29; no voters:
I20260807 03:05:47.307788 27883 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 03:05:47.307937 27889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 03:05:47.309075 27889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [term 3 LEADER]: Becoming Leader. State: Replica: 1b0d7bd5b2ba4d00ab6615b6ed9caa29, State: Running, Role: LEADER
I20260807 03:05:47.309602 27889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } }
I20260807 03:05:47.309979 27883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:05:47.312014 27891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0d7bd5b2ba4d00ab6615b6ed9caa29. Latest consensus state: current_term: 3 leader_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } } }
I20260807 03:05:47.312346 27891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:47.312824 27890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0d7bd5b2ba4d00ab6615b6ed9caa29" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 39491 } } }
I20260807 03:05:47.313079 27890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0d7bd5b2ba4d00ab6615b6ed9caa29 [sys.catalog]: This master's current role is: LEADER
I20260807 03:05:47.324791 27895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:05:47.333735 27895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:05:47.335244 27895 catalog_manager.cc:1259] Loaded cluster ID: cca53674857548b7b8b847b5e28a8b23
I20260807 03:05:47.335505 27895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 03:05:47.337616 27895 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa652f089ad google::LogMessage::Fail() at ??:0
@ 0x7fa652f0b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa652f084de google::LogMessage::Flush() at ??:0
@ 0x7fa652f09a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260807 03:05:47.351075 27915 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fa66893b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa668938976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa6689c0610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa6689c0439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa655b54345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa655b5791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa655b57799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa655b32eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa658e0c6db start_thread at ??:0
@ 0x7fa651da261f clone at ??:0
W20260807 03:05:47.643488 27885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:47.643754 27885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:47.643852 27885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:47.650903 27885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:47.651015 27885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:47.651135 27885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:05:47.660897 27885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:44881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=38383
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:47.662202 27885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:47.663524 27885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:47.675892 27946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:47.675956 27949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:47.675931 27947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:47.676764 27885 server_base.cc:1034] running on GCE node
I20260807 03:05:47.677356 27885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:47.678247 27885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:47.679458 27885 hybrid_clock.cc:648] HybridClock initialized: now 1786071947679415 us; error 35 us; skew 500 ppm
I20260807 03:05:47.679762 27885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:47 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:47.682758 27885 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:05:47.684074 27885 webserver.cc:460] Webserver started at http://127.5.122.1:38383/ using document root <none> and password file <none>
I20260807 03:05:47.684696 27885 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:47.684794 27885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:47.689929 27885 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:05:47.692744 27958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:47.694024 27885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:05:47.694187 27885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "134a1de9819747159579124934a0a554"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "216fdeb6c4c828327f54d7277475c15b"
server_key_iv: "9c27b1c09dd83b704ce3e31f0158de99"
server_key_version: "encryptionkey@0"
I20260807 03:05:47.695176 27885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:47.711947 27885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:47.716168 27885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:47.716441 27885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:47.717391 27885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:47.719354 27970 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 03:05:47.724773 27885 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 03:05:47.724915 27885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 03:05:47.725025 27885 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 03:05:47.728163 27885 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 03:05:47.728238 27885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.002s
I20260807 03:05:47.728544 27970 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Bootstrap starting.
I20260807 03:05:47.736543 27970 log.cc:824] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Log is configured to *not* fsync() on all Append() calls
I20260807 03:05:47.751178 27970 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:05:47.751982 27970 tablet_bootstrap.cc:492] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Bootstrap complete.
I20260807 03:05:47.752452 27970 ts_tablet_manager.cc:1397] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.012s
I20260807 03:05:47.761375 27970 raft_consensus.cc:348] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:47.762830 27970 raft_consensus.cc:729] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134a1de9819747159579124934a0a554, State: Initialized, Role: FOLLOWER
I20260807 03:05:47.763489 27970 consensus_queue.cc:260] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:47.765056 27970 ts_tablet_manager.cc:1428] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 03:05:47.769619 27885 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:44881
I20260807 03:05:47.769696 28130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:44881 every 8 connection(s)
I20260807 03:05:47.771052 27885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:05:47.774158 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 27885
I20260807 03:05:47.774809 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:39527
--local_ip_for_outbound_sockets=127.5.122.2
--tserver_master_addrs=127.5.122.62:39491
--webserver_port=34873
--webserver_interface=127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:05:47.775624 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:47.773559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:37649)
0807 03:05:47.774432 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:47.774692 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":357,"threads_started":1}
W20260807 03:05:47.776186 28134 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:39491 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.62:39491: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
I20260807 03:05:47.777002 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:47.776698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:55961)
0807 03:05:47.776835 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:47.776903 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 03:05:47.777774 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:47.777480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:40315)
0807 03:05:47.777594 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:47.777662 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 03:05:47.778002 28134 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 03:05:48.093907 28139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:48.094182 28139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:48.094326 28139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:48.101604 28139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:48.101717 28139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:48.101884 28139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:05:48.111410 28139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:39527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=34873
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:48.112633 28139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:48.113950 28139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:48.126343 28149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:48.126444 28147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:48.126338 28151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:48.127079 28139 server_base.cc:1034] running on GCE node
I20260807 03:05:48.127614 28139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:48.128479 28139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:48.129663 28139 hybrid_clock.cc:648] HybridClock initialized: now 1786071948129612 us; error 47 us; skew 500 ppm
I20260807 03:05:48.129921 28139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:48 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071948, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:48.132855 28139 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:05:48.134189 28139 webserver.cc:460] Webserver started at http://127.5.122.2:34873/ using document root <none> and password file <none>
I20260807 03:05:48.134838 28139 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:48.134938 28139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:48.139907 28139 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 03:05:48.142428 28161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:48.143680 28139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:05:48.143810 28139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "b748f970ed9f494daf27b1cabcafcdc3"
format_stamp: "Formatted at 2026-08-07 03:05:42 on dist-test-slave-fgb3"
server_key: "d95113664e19273dc25b2e299c476303"
server_key_iv: "1cddd782e4818fe9a33a864300e52259"
server_key_version: "encryptionkey@0"
I20260807 03:05:48.144727 28139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:48.159770 28139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:48.163447 28139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:48.163672 28139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:48.164561 28139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:48.165814 28139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:48.165906 28139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:48.165984 28139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:48.166042 28139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:48.196058 28139 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:39527
I20260807 03:05:48.196115 28330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:39527 every 8 connection(s)
I20260807 03:05:48.197499 28139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:05:48.199703 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 28139
I20260807 03:05:48.200325 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:36627
--local_ip_for_outbound_sockets=127.5.122.3
--tserver_master_addrs=127.5.122.62:39491
--webserver_port=41043
--webserver_interface=127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:05:48.202625 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:48.200402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:44395)
0807 03:05:48.201230 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:48.201483 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":294,"threads_started":1}
W20260807 03:05:48.203315 28331 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:39491 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.62:39491: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
I20260807 03:05:48.204133 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:48.203786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:33597)
0807 03:05:48.203931 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:48.204011 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 03:05:48.204967 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:48.204629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:48401)
0807 03:05:48.204758 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:48.204842 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 03:05:48.205180 28331 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 03:05:48.519938 28336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:05:48.520180 28336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:05:48.520308 28336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:05:48.527252 28336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:05:48.527349 28336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:05:48.527503 28336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:05:48.536651 28336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:36627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=41043
--tserver_master_addrs=127.5.122.62:39491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:05:48.537807 28336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:05:48.539083 28336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:05:48.551198 28350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:48.551267 28346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:05:48.551244 28347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:05:48.551975 28336 server_base.cc:1034] running on GCE node
I20260807 03:05:48.552500 28336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:05:48.553330 28336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:05:48.554522 28336 hybrid_clock.cc:648] HybridClock initialized: now 1786071948554471 us; error 40 us; skew 500 ppm
I20260807 03:05:48.554797 28336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:05:48 dist-test-slave-fgb3 krb5kdc[26135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071948, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:05:48.557626 28336 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:05:48.559005 28336 webserver.cc:460] Webserver started at http://127.5.122.3:41043/ using document root <none> and password file <none>
I20260807 03:05:48.559648 28336 fs_manager.cc:359] Metadata directory not provided
I20260807 03:05:48.559765 28336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:05:48.564731 28336 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 03:05:48.567309 28362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:48.568503 28336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:05:48.568646 28336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "749cb52b4c0d40b2bacd3e647cc227b9"
format_stamp: "Formatted at 2026-08-07 03:05:43 on dist-test-slave-fgb3"
server_key: "85f75fd20c436ddaebb50393ffc19ebf"
server_key_iv: "5aa0a4fef66ae5061d2ebdb3022bf787"
server_key_version: "encryptionkey@0"
I20260807 03:05:48.569625 28336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:05:48.588786 28336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:05:48.592514 28336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:05:48.592767 28336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:05:48.593729 28336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:05:48.595057 28336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:05:48.595153 28336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:48.595260 28336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:05:48.595327 28336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:05:48.625362 28336 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:36627
I20260807 03:05:48.625406 28538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:36627 every 8 connection(s)
I20260807 03:05:48.626802 28336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:05:48.631292 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:48.629538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:38357)
0807 03:05:48.630227 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:48.630472 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":275,"threads_started":1}
W20260807 03:05:48.631880 28539 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:39491 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.62:39491: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
I20260807 03:05:48.632597 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:48.632293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:41625)
0807 03:05:48.632423 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:48.632497 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 03:05:48.633291 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:48.633023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:56043)
0807 03:05:48.633123 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:48.633190 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 03:05:48.633499 28539 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 03:05:48.635994 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 28336
I20260807 03:05:48.779223 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:48.778532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:51303)
0807 03:05:48.778935 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:48.779020 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":184,"threads_started":1}
I20260807 03:05:49.206492 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:49.205682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:57201)
0807 03:05:49.206118 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:49.206225 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":207,"threads_started":1}
I20260807 03:05:49.597180 28551 raft_consensus.cc:482] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:49.597478 28551 raft_consensus.cc:504] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:49.599088 28551 leader_election.cc:288] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527), 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
Aug 07 03:05:49 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
Aug 07 03:05:49 dist-test-slave-fgb3 krb5kdc[26135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:05:49.620965 28552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:49.599381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39527 (local address 127.5.122.1:59761)
0807 03:05:49.599962 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:49.600018 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:49.600610 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:49.604353 (+ 3743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:49.604362 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:49.605261 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:49.605993 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:49.606012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.607540 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.607544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:49.608934 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:49.608946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.609889 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.609897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:49.610158 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:49.610972 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:49.610991 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:49.614072 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:49.617365 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:49.617374 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:49.617389 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:49.617898 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:49.618373 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:49.618378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:49.618382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:49.618582 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:49.619371 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:49.619375 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:49.619693 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 03:05:49.619961 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 03:05:49.620507 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":201,"spinlock_wait_cycles":24064,"thread_start_us":269,"threads_started":1}
I20260807 03:05:49.621068 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:49.599661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:59761 (local address 127.5.122.2:39527)
0807 03:05:49.601720 (+ 2059us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:49.601735 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:49.601781 (+ 46us) server_negotiation.cc:407] Connection header received
0807 03:05:49.603460 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:49.603471 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:49.603912 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:49.604203 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:49.606174 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.607353 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.609070 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.609669 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.609816 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:49.614366 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:49.614385 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:49.614399 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:49.614445 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:49.617222 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:49.618038 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:49.618047 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:49.618055 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:49.618220 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:49.618690 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:49.618695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:49.618698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:49.619209 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:49.619383 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:49.620306 (+ 923us) server_negotiation.cc:299] Negotiation successful
0807 03:05:49.620666 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1859,"thread_start_us":1677,"threads_started":1}
I20260807 03:05:49.621270 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:49.600400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:52935 (local address 127.5.122.3:36627)
0807 03:05:49.601905 (+ 1505us) server_negotiation.cc:206] Beginning negotiation
0807 03:05:49.601915 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:05:49.601944 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:05:49.603568 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:05:49.603578 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:05:49.603944 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:05:49.604228 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:05:49.606175 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.607357 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.609069 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.609687 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.609868 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:49.614124 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:05:49.614135 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:05:49.614154 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:05:49.614208 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:05:49.617294 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:49.618054 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:49.618061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:49.618066 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:49.618220 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:05:49.618686 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:05:49.618691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:05:49.618694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:05:49.619205 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:05:49.619371 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 03:05:49.620491 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0807 03:05:49.620885 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1366,"thread_start_us":1260,"threads_started":1}
I20260807 03:05:49.621063 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:49.600051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:36627 (local address 127.5.122.1:52935)
0807 03:05:49.601386 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:49.601403 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:05:49.601507 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:05:49.604350 (+ 2843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:05:49.604362 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:05:49.605261 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:05:49.605993 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:49.606012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.607530 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.607535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:49.608930 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:05:49.608943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:05:49.609915 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:05:49.609923 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:05:49.610156 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:05:49.610908 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:05:49.610943 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:05:49.613799 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:05:49.617427 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:49.617431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:49.617433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:49.617912 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:49.618361 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:05:49.618367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:05:49.618371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:05:49.618576 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:05:49.619359 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:05:49.619366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:05:49.619704 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 03:05:49.619961 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 03:05:49.620678 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"mutex_wait_us":385,"spinlock_wait_cycles":17280,"thread_start_us":1020,"threads_started":1}
W20260807 03:05:49.625190 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
W20260807 03:05:49.625525 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:49.625701 27961 leader_election.cc:302] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 134a1de9819747159579124934a0a554; no voters: 749cb52b4c0d40b2bacd3e647cc227b9, b748f970ed9f494daf27b1cabcafcdc3
I20260807 03:05:49.626144 28551 raft_consensus.cc:2726] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:05:49.634778 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:49.633960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:58511)
0807 03:05:49.634429 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:49.634558 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":212,"threads_started":1}
I20260807 03:05:49.780404 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:49.780005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:49037)
0807 03:05:49.780168 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:49.780265 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 03:05:50.208010 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:50.207311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:50675)
0807 03:05:50.207770 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:50.207852 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":251,"threads_started":1}
I20260807 03:05:50.636428 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:50.635653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:41219)
0807 03:05:50.636146 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:50.636244 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":312,"threads_started":1}
I20260807 03:05:50.782014 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:50.781275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:56061)
0807 03:05:50.781731 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:50.781829 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":227,"threads_started":1}
I20260807 03:05:51.209565 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:51.208852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:41803)
0807 03:05:51.209291 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:51.209373 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":252,"threads_started":1}
I20260807 03:05:51.637889 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:51.637180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:40679)
0807 03:05:51.637651 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:51.637729 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":226,"threads_started":1}
I20260807 03:05:51.689810 28568 raft_consensus.cc:482] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:51.690007 28568 raft_consensus.cc:504] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:51.690685 28568 leader_election.cc:288] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527), 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
W20260807 03:05:51.691699 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
W20260807 03:05:51.691897 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:51.691973 27961 leader_election.cc:302] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 134a1de9819747159579124934a0a554; no voters: 749cb52b4c0d40b2bacd3e647cc227b9, b748f970ed9f494daf27b1cabcafcdc3
I20260807 03:05:51.692191 28568 raft_consensus.cc:2726] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:05:51.783663 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:51.782910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:51921)
0807 03:05:51.783366 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:51.783469 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":200,"threads_started":1}
I20260807 03:05:52.211228 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:52.210511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:48547)
0807 03:05:52.210932 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:52.211026 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":202,"threads_started":1}
I20260807 03:05:52.639348 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:52.638694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:51145)
0807 03:05:52.639096 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:52.639171 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":192,"threads_started":1}
I20260807 03:05:52.785254 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:52.784535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:43605)
0807 03:05:52.784993 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:52.785069 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":205,"threads_started":1}
I20260807 03:05:53.212764 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:53.212099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:51745)
0807 03:05:53.212516 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:53.212595 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":224,"threads_started":1}
I20260807 03:05:53.640792 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:53.640117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:54977)
0807 03:05:53.640557 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:53.640632 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":250,"threads_started":1}
I20260807 03:05:53.786677 28577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:53.786020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:47909)
0807 03:05:53.786450 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:53.786526 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":205,"threads_started":1}
I20260807 03:05:54.214305 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:54.213580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:50681)
0807 03:05:54.214043 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:54.214122 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":246,"threads_started":1}
I20260807 03:05:54.642253 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:54.641548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:59835)
0807 03:05:54.641967 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:54.642060 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":226,"threads_started":1}
I20260807 03:05:54.788087 28581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:54.787412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:35049)
0807 03:05:54.787819 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:54.787894 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":208,"threads_started":1}
I20260807 03:05:55.000722 28582 raft_consensus.cc:482] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:55.000906 28582 raft_consensus.cc:504] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:55.001467 28582 leader_election.cc:288] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527), 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
W20260807 03:05:55.002421 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
W20260807 03:05:55.002617 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:55.002702 27961 leader_election.cc:302] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 134a1de9819747159579124934a0a554; no voters: 749cb52b4c0d40b2bacd3e647cc227b9, b748f970ed9f494daf27b1cabcafcdc3
I20260807 03:05:55.002933 28582 raft_consensus.cc:2726] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:05:55.215893 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:55.215113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:54031)
0807 03:05:55.215616 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:55.215707 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":230,"threads_started":1}
I20260807 03:05:55.643832 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:55.643112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:56171)
0807 03:05:55.643576 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:55.643655 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":216,"threads_started":1}
I20260807 03:05:55.789495 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:55.788843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:34065)
0807 03:05:55.789257 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:55.789336 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":205,"threads_started":1}
I20260807 03:05:56.217406 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:56.216718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:59961)
0807 03:05:56.217126 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:56.217211 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":206,"threads_started":1}
I20260807 03:05:56.645253 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:56.644602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:35613)
0807 03:05:56.644997 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:56.645073 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":200,"threads_started":1}
I20260807 03:05:56.791103 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:56.790387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:32821)
0807 03:05:56.790842 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:56.790935 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":219,"threads_started":1}
I20260807 03:05:57.108659 28599 raft_consensus.cc:482] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:57.108875 28599 raft_consensus.cc:504] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:57.109521 28599 leader_election.cc:288] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527), 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
W20260807 03:05:57.110456 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
W20260807 03:05:57.110659 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:57.110741 27961 leader_election.cc:302] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 134a1de9819747159579124934a0a554; no voters: 749cb52b4c0d40b2bacd3e647cc227b9, b748f970ed9f494daf27b1cabcafcdc3
I20260807 03:05:57.110944 28599 raft_consensus.cc:2726] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:05:57.219168 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:57.218388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:33623)
0807 03:05:57.218889 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:57.219004 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":234,"threads_started":1}
I20260807 03:05:57.646936 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:57.646195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:41279)
0807 03:05:57.646695 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:57.646782 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":250,"threads_started":1}
I20260807 03:05:57.792644 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:57.791973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:44695)
0807 03:05:57.792398 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:57.792476 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":218,"threads_started":1}
I20260807 03:05:58.220745 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:58.220036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:52229)
0807 03:05:58.220478 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:58.220562 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":228,"threads_started":1}
I20260807 03:05:58.648522 28607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:58.647801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:49291)
0807 03:05:58.648252 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:58.648338 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":247,"threads_started":1}
I20260807 03:05:58.794083 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:58.793417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:37837)
0807 03:05:58.793839 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:58.793915 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":213,"threads_started":1}
I20260807 03:05:59.222431 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:59.221566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:49409)
0807 03:05:59.222084 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:59.222168 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":260,"threads_started":1}
I20260807 03:05:59.499060 28611 raft_consensus.cc:482] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:05:59.499228 28611 raft_consensus.cc:504] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748f970ed9f494daf27b1cabcafcdc3" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39527 } } peers { permanent_uuid: "134a1de9819747159579124934a0a554" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44881 } } peers { permanent_uuid: "749cb52b4c0d40b2bacd3e647cc227b9" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 36627 } }
I20260807 03:05:59.499879 28611 leader_election.cc:288] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527), 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627)
W20260807 03:05:59.500800 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b748f970ed9f494daf27b1cabcafcdc3 (127.5.122.2:39527): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
W20260807 03:05:59.500984 27961 leader_election.cc:341] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 749cb52b4c0d40b2bacd3e647cc227b9 (127.5.122.3:36627): Not found: Tablet not found: 6bafab2c296548f9bd1ab51c65824956
I20260807 03:05:59.501080 27961 leader_election.cc:302] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 134a1de9819747159579124934a0a554; no voters: 749cb52b4c0d40b2bacd3e647cc227b9, b748f970ed9f494daf27b1cabcafcdc3
I20260807 03:05:59.501298 28611 raft_consensus.cc:2726] T 6bafab2c296548f9bd1ab51c65824956 P 134a1de9819747159579124934a0a554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:05:59.649987 28612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:59.649282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:41603)
0807 03:05:59.649731 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:59.649812 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":218,"threads_started":1}
I20260807 03:05:59.795572 28613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:05:59.794831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:41133)
0807 03:05:59.795276 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 03:05:59.795365 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":207,"threads_started":1}
I20260807 03:06:00.223985 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:00.223307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:40107)
0807 03:06:00.223740 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:00.223817 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":224,"threads_started":1}
I20260807 03:06:00.651365 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:00.650723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:33631)
0807 03:06:00.651134 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:00.651207 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":218,"threads_started":1}
I20260807 03:06:00.797098 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:00.796386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:34461)
0807 03:06:00.796808 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:00.796908 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":219,"threads_started":1}
I20260807 03:06:01.225466 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:01.224760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:34085)
0807 03:06:01.225221 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:01.225309 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":237,"threads_started":1}
I20260807 03:06:01.652927 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:01.652143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:47719)
0807 03:06:01.652651 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:01.652758 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":273,"threads_started":1}
I20260807 03:06:01.798838 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:01.797954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:43927)
0807 03:06:01.798501 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:01.798620 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":242,"threads_started":1}
I20260807 03:06:02.227149 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:02.226409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:57421)
0807 03:06:02.226875 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:02.226958 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":234,"threads_started":1}
I20260807 03:06:02.654480 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:02.653718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:37289)
0807 03:06:02.654176 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:02.654257 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":245,"threads_started":1}
I20260807 03:06:02.800520 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:02.799855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.1:57251)
0807 03:06:02.800272 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:02.800345 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":213,"threads_started":1}
I20260807 03:06:03.228861 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:03.228159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.2:42811)
0807 03:06:03.228592 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:03.228703 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":222,"threads_started":1}
I20260807 03:06:03.645534 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 27885
I20260807 03:06:03.656168 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:03.655458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:39491 (local address 127.5.122.3:38839)
0807 03:06:03.655898 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:03.656019 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:39491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
I20260807 03:06:03.659675 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:03.646444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.5.122.1:44881)
0807 03:06:03.646925 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:03.646936 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:03.646977 (+ 41us) server_negotiation.cc:407] Connection header received
0807 03:06:03.647044 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:03.647053 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:03.647326 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:03.647482 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:03.649456 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:03.650492 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:03.651661 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:03.652200 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:03.652290 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:03.654673 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:03.654682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:03.654694 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:03.654722 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:03.657385 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:03.658015 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:03.658021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:03.658023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:03.658142 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:03.658445 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:03.658448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:03.658450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:03.658762 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:03.658973 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:03.659190 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 03:06:03.659372 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":205,"threads_started":1}
I20260807 03:06:03.660774 28126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:03.746378 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 27885
I20260807 03:06:03.771842 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 28139
I20260807 03:06:03.784847 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:03.772629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.5.122.2:39527)
0807 03:06:03.773136 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:03.773141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:03.773155 (+ 14us) server_negotiation.cc:407] Connection header received
0807 03:06:03.773210 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:03.773214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:03.773269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:03.773363 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:03.774905 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:03.775701 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:03.776955 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:03.777369 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:03.777487 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:03.780043 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:03.780047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:03.780051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:03.780083 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:03.782360 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:03.783001 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:03.783007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:03.783010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:03.783134 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:03.783471 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:03.783476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:03.783480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:03.783736 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:03.783974 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:03.784259 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 03:06:03.784475 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":271,"threads_started":1}
I20260807 03:06:03.785537 28324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:03.867383 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 28139
I20260807 03:06:03.888852 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 28336
I20260807 03:06:03.901036 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:03.889592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49622 (local address 127.5.122.3:36627)
0807 03:06:03.890106 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:03.890111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:03.890128 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:06:03.890181 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:03.890185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:03.890252 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:03.890383 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:03.891621 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:03.892358 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:03.893538 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:03.893935 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:03.894021 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:03.896494 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:03.896499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:03.896502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:03.896534 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:03.898929 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:03.899511 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:03.899514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:03.899516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:03.899655 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:03.899923 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:03.899926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:03.899928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:03.900157 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:03.900363 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:03.900524 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 03:06:03.900676 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":255,"threads_started":1}
I20260807 03:06:03.901671 28530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:03.981734 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 28336
2026-08-07T03:06:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24213 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:06:04 dist-test-slave-fgb3 krb5kdc[28640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:06:04 dist-test-slave-fgb3 krb5kdc[28640](info): set up 2 sockets
Aug 07 03:06:04 dist-test-slave-fgb3 krb5kdc[28640](info): commencing operation
krb5kdc: starting...
W20260807 03:06:05.186126 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:06:05 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:06:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:06:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:06:05.455317 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33843
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:06:05.779394 28658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:05.779711 28658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:05.779852 28658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:05.786973 28658 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 03:06:05.787063 28658 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 03:06:05.787123 28658 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 03:06:05.787194 28658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:06:05.787277 28658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:05.787370 28658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:05.787426 28658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:06:05.787500 28658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:06:05.796969 28658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:05.798236 28658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:05.799526 28658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:05.810599 28665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:05.810632 28668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:05.810810 28664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:05.811431 28658 server_base.cc:1034] running on GCE node
I20260807 03:06:05.812103 28658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:05.813344 28658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:05.814545 28658 hybrid_clock.cc:648] HybridClock initialized: now 1786071965814480 us; error 35 us; skew 500 ppm
I20260807 03:06:05.814821 28658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:05 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071965, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:05.817732 28658 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:06:05.819079 28658 webserver.cc:460] Webserver started at http://127.5.122.62:33687/ using document root <none> and password file <none>
I20260807 03:06:05.819693 28658 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:05.819777 28658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:05.820027 28658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:06:05.822846 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "2acc297bdaa2448e86dd581d3e1f3b82"
format_stamp: "Formatted at 2026-08-07 03:06:05 on dist-test-slave-fgb3"
server_key: "b562e6e0ebd4992af1f95d70884637bb"
server_key_iv: "96a1979ea560696f60d7622ac0120ffd"
server_key_version: "encryptionkey@0"
I20260807 03:06:05.823536 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "2acc297bdaa2448e86dd581d3e1f3b82"
format_stamp: "Formatted at 2026-08-07 03:06:05 on dist-test-slave-fgb3"
server_key: "b562e6e0ebd4992af1f95d70884637bb"
server_key_iv: "96a1979ea560696f60d7622ac0120ffd"
server_key_version: "encryptionkey@0"
I20260807 03:06:05.828399 28658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:06:05.831634 28683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:05.832841 28658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:06:05.832973 28658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "2acc297bdaa2448e86dd581d3e1f3b82"
format_stamp: "Formatted at 2026-08-07 03:06:05 on dist-test-slave-fgb3"
server_key: "b562e6e0ebd4992af1f95d70884637bb"
server_key_iv: "96a1979ea560696f60d7622ac0120ffd"
server_key_version: "encryptionkey@0"
I20260807 03:06:05.833146 28658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:05.846681 28658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:05.850850 28658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:05.851084 28658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:05.868106 28658 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:33843
I20260807 03:06:05.868109 28758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:33843 every 8 connection(s)
I20260807 03:06:05.869518 28658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:06:05.870720 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 28658
I20260807 03:06:05.870952 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:06:05.871795 5608 external_mini_cluster.cc:1428] Setting key 9f48cccac1feb300dbd3775aa26c1d91
I20260807 03:06:05.874104 28759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 03:06:05 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:06:05.886706 28759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Bootstrap starting.
I20260807 03:06:05.889622 28759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Neither blocks nor log segments found. Creating new log.
I20260807 03:06:05.890645 28759 log.cc:824] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:05.893844 28759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: No bootstrap required, opened a new log
I20260807 03:06:05.894639 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:05.873834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.5.122.62:33843)
0807 03:06:05.874610 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:05.874621 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:05.874661 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:06:05.876262 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:05.876283 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:05.876651 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:05.876961 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:05.878799 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:05.880089 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:05.881934 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:05.882622 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:05.882763 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:05.887341 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:05.887348 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:05.887360 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:05.887393 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:05.890196 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:05.890898 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:05.890906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:05.890914 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:05.891134 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:05.891578 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:05.891581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:05.891583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:05.892203 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:05.892493 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:05.892867 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 03:06:05.893312 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":336,"threads_started":1}
I20260807 03:06:05.901289 28759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:05.901636 28759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:06:05.901710 28759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acc297bdaa2448e86dd581d3e1f3b82, State: Initialized, Role: FOLLOWER
I20260807 03:06:05.902206 28759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:05.902460 28759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:06:05.902585 28759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:06:05.902729 28759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:06:05.904898 28759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:05.905364 28759 leader_election.cc:302] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2acc297bdaa2448e86dd581d3e1f3b82; no voters:
I20260807 03:06:05.905912 28759 leader_election.cc:288] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:06:05.906015 28767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:06:05.906664 28767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 LEADER]: Becoming Leader. State: Replica: 2acc297bdaa2448e86dd581d3e1f3b82, State: Running, Role: LEADER
I20260807 03:06:05.907172 28767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:05.907863 28759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:06:05.909543 28769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } } }
I20260807 03:06:05.909933 28769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: This master's current role is: LEADER
I20260807 03:06:05.911164 28770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2acc297bdaa2448e86dd581d3e1f3b82. Latest consensus state: current_term: 1 leader_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } } }
I20260807 03:06:05.911525 28770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: This master's current role is: LEADER
I20260807 03:06:05.913049 28776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:06:05.917683 28776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:06:05.925498 28776 catalog_manager.cc:1347] Generated new cluster ID: 39be175e8dbd46dcac018ae0629ef998
I20260807 03:06:05.925613 28776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:06:05.973769 28776 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:06:05.975688 28776 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:06:05.985097 28776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Generated new TSK 0
I20260807 03:06:05.986099 28776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:06:06.052001 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33843
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:06:06.371192 28795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:06.371480 28795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:06.371626 28795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:06.378686 28795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:06.378803 28795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:06.378957 28795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:06:06.388450 28795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:06.389720 28795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:06.391007 28795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:06.403299 28806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:06.403255 28803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:06.403326 28802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:06.404049 28795 server_base.cc:1034] running on GCE node
I20260807 03:06:06.404606 28795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:06.405505 28795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:06.406714 28795 hybrid_clock.cc:648] HybridClock initialized: now 1786071966406665 us; error 43 us; skew 500 ppm
I20260807 03:06:06.406997 28795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:06 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:06.410069 28795 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:06:06.411432 28795 webserver.cc:460] Webserver started at http://127.5.122.1:40521/ using document root <none> and password file <none>
I20260807 03:06:06.412418 28795 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:06.412526 28795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:06.412850 28795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:06:06.415728 28795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "76c3712a0d23410f8c6bfda2b8b2db96"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4a344f75afc414f35c40b605d4cc43ae"
server_key_iv: "1b75342e2220fe67583fb463ee299499"
server_key_version: "encryptionkey@0"
I20260807 03:06:06.416407 28795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "76c3712a0d23410f8c6bfda2b8b2db96"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4a344f75afc414f35c40b605d4cc43ae"
server_key_iv: "1b75342e2220fe67583fb463ee299499"
server_key_version: "encryptionkey@0"
I20260807 03:06:06.421218 28795 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 03:06:06.424434 28819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:06.425666 28795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:06:06.425796 28795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "76c3712a0d23410f8c6bfda2b8b2db96"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4a344f75afc414f35c40b605d4cc43ae"
server_key_iv: "1b75342e2220fe67583fb463ee299499"
server_key_version: "encryptionkey@0"
I20260807 03:06:06.425963 28795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:06.467275 28795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:06.470954 28795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:06.471196 28795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:06.472082 28795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:06.473335 28795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:06.473407 28795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:06.473505 28795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:06.473547 28795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:06.503336 28795 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:37953
I20260807 03:06:06.503386 28980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:37953 every 8 connection(s)
I20260807 03:06:06.504544 28795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:06:06.509953 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 28795
I20260807 03:06:06.510159 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:06:06.510922 5608 external_mini_cluster.cc:1428] Setting key 601e655f85ee3ed9766a9c2ffee66984
Aug 07 03:06:06 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:06:06.523875 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:06.506935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:39421 (local address 127.5.122.62:33843)
0807 03:06:06.507365 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:06.507369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:06.508208 (+ 839us) server_negotiation.cc:407] Connection header received
0807 03:06:06.510179 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:06.510185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:06.510275 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:06.510429 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:06.511933 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:06.512732 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:06.514277 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:06.514899 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:06.515032 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:06.518199 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:06.518205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:06.518209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:06.518243 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:06.521126 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:06.521754 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:06.521759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:06.521762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:06.521901 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:06.522253 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:06.522257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:06.522259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:06.522589 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:06.522719 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:06.523332 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 03:06:06.523547 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":221,"threads_started":1}
I20260807 03:06:06.524652 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:06.507363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:39421)
0807 03:06:06.508061 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:06.508099 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:06.509905 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:06.510598 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:06.510606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:06.511132 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:06.511760 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:06.511780 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:06.512902 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:06.512905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:06.514151 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:06.514163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:06.515095 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:06.515100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:06.515279 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:06.515911 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:06.515958 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:06.517985 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:06.521241 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:06.521247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:06.521258 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:06.521648 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:06.521990 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:06.521993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:06.521995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:06.522161 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:06.522700 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:06.522706 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:06.522992 (+ 286us) client_negotiation.cc:769] Sending connection context
0807 03:06:06.523207 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 03:06:06.523559 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":249,"threads_started":1}
I20260807 03:06:06.526077 28981 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33843
I20260807 03:06:06.526430 28981 heartbeater.cc:464] Registering TS with master...
I20260807 03:06:06.527118 28981 heartbeater.cc:511] Master 127.5.122.62:33843 requested a full tablet report, sending...
I20260807 03:06:06.529361 28705 ts_manager.cc:194] Registered new tserver with Master: 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953)
I20260807 03:06:06.531209 28705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:39421
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:06:06.570675 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33843
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:06:06.892817 28988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:06.893080 28988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:06.893203 28988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:06.900261 28988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:06.900372 28988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:06.900518 28988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:06:06.909915 28988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:06.911087 28988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:06.912317 28988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:06.924652 28993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:06.924652 28994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:06.924729 28996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:06.925725 28988 server_base.cc:1034] running on GCE node
I20260807 03:06:06.926282 28988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:06.927174 28988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:06.928359 28988 hybrid_clock.cc:648] HybridClock initialized: now 1786071966928311 us; error 40 us; skew 500 ppm
I20260807 03:06:06.928639 28988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:06 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:06.936841 28988 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:06:06.938236 28988 webserver.cc:460] Webserver started at http://127.5.122.2:37909/ using document root <none> and password file <none>
I20260807 03:06:06.938879 28988 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:06.938989 28988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:06.939242 28988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:06:06.941922 28988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "33a975be89644b7bb1cf3d9c41afac84"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4f571a779c5e1acf5fb5e30266593133"
server_key_iv: "0cda1ccff40c344d0be1dab68f55d57a"
server_key_version: "encryptionkey@0"
I20260807 03:06:06.942657 28988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "33a975be89644b7bb1cf3d9c41afac84"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4f571a779c5e1acf5fb5e30266593133"
server_key_iv: "0cda1ccff40c344d0be1dab68f55d57a"
server_key_version: "encryptionkey@0"
I20260807 03:06:06.947408 28988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 03:06:06.950562 29005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:06.951750 28988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:06:06.951894 28988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "33a975be89644b7bb1cf3d9c41afac84"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4f571a779c5e1acf5fb5e30266593133"
server_key_iv: "0cda1ccff40c344d0be1dab68f55d57a"
server_key_version: "encryptionkey@0"
I20260807 03:06:06.952081 28988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:06.971676 28988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:06.975557 28988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:06.975821 28988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:06.976784 28988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:06.978258 28988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:06.978363 28988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 03:06:06.978488 28988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:06.978545 28988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:07.008605 28988 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:37601
I20260807 03:06:07.008666 29164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:37601 every 8 connection(s)
I20260807 03:06:07.009927 28988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:06:07.014649 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 28988
I20260807 03:06:07.014827 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:06:07.015266 5608 external_mini_cluster.cc:1428] Setting key 657d305db67430e5759fc9284c731b19
Aug 07 03:06:07 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:06:07.029069 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.012266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:46157 (local address 127.5.122.62:33843)
0807 03:06:07.012425 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.012429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.013600 (+ 1171us) server_negotiation.cc:407] Connection header received
0807 03:06:07.015513 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.015518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.015604 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.015692 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:07.017398 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.018089 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.019623 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.020053 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.020162 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.023532 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:07.023537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:07.023541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:07.023572 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:07.026151 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.026864 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.026868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.026870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.026990 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.027402 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.027406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.027408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.027658 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:07.027786 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.028482 (+ 696us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.028710 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 03:06:07.030040 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.012724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:46157)
0807 03:06:07.013445 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:07.013490 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:07.015251 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:07.015935 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:07.015942 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:07.016561 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:07.017228 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.017245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.018236 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.018240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.019486 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.019498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.020272 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.020280 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.020515 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.021124 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:07.021151 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:07.023315 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:07.026286 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:07.026295 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:07.026332 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:07.026745 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:07.027092 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:07.027098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:07.027102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:07.027283 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:07.027780 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:07.027788 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:07.028106 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 03:06:07.028343 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 03:06:07.028895 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":270,"threads_started":1}
I20260807 03:06:07.031445 29165 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33843
I20260807 03:06:07.031750 29165 heartbeater.cc:464] Registering TS with master...
I20260807 03:06:07.032536 29165 heartbeater.cc:511] Master 127.5.122.62:33843 requested a full tablet report, sending...
I20260807 03:06:07.034092 28705 ts_manager.cc:194] Registered new tserver with Master: 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601)
I20260807 03:06:07.035171 28705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:46157
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:06:07.074846 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33843
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:06:07.392315 29171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:07.392596 29171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:07.392694 29171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:07.399816 29171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:07.399926 29171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:07.400049 29171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:06:07.409435 29171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:07.410610 29171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:07.411792 29171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:07.424042 29183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:07.424063 29182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:07.424351 29185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:07.424732 29171 server_base.cc:1034] running on GCE node
I20260807 03:06:07.425298 29171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:07.426175 29171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:07.427376 29171 hybrid_clock.cc:648] HybridClock initialized: now 1786071967427322 us; error 46 us; skew 500 ppm
I20260807 03:06:07.427676 29171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:07 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071967, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:07.430769 29171 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:06:07.432024 29171 webserver.cc:460] Webserver started at http://127.5.122.3:42799/ using document root <none> and password file <none>
I20260807 03:06:07.432627 29171 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:07.432725 29171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:07.432986 29171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:06:07.435746 29171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "40c33d52aab44873a803fc40e306d5a4"
format_stamp: "Formatted at 2026-08-07 03:06:07 on dist-test-slave-fgb3"
server_key: "698549b05a7d897a13d1806f7ea1c965"
server_key_iv: "78ff8ba02484bcbd6a12d1f64a528281"
server_key_version: "encryptionkey@0"
I20260807 03:06:07.436542 29171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "40c33d52aab44873a803fc40e306d5a4"
format_stamp: "Formatted at 2026-08-07 03:06:07 on dist-test-slave-fgb3"
server_key: "698549b05a7d897a13d1806f7ea1c965"
server_key_iv: "78ff8ba02484bcbd6a12d1f64a528281"
server_key_version: "encryptionkey@0"
I20260807 03:06:07.441385 29171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 03:06:07.444597 29195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:07.445756 29171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 03:06:07.445889 29171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "40c33d52aab44873a803fc40e306d5a4"
format_stamp: "Formatted at 2026-08-07 03:06:07 on dist-test-slave-fgb3"
server_key: "698549b05a7d897a13d1806f7ea1c965"
server_key_iv: "78ff8ba02484bcbd6a12d1f64a528281"
server_key_version: "encryptionkey@0"
I20260807 03:06:07.446041 29171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:07.467450 29171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:07.471131 29171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:07.471367 29171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:07.472218 29171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:07.473424 29171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:07.473500 29171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:07.473587 29171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:07.473627 29171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:07.503192 29171 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:34269
I20260807 03:06:07.503289 29351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:34269 every 8 connection(s)
I20260807 03:06:07.504411 29171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:06:07.509562 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 29171
I20260807 03:06:07.509743 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:06:07.510174 5608 external_mini_cluster.cc:1428] Setting key 43af639a7057a35039fbaa45548be34f
Aug 07 03:06:07 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071967, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:06:07.523286 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.506850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:35579 (local address 127.5.122.62:33843)
0807 03:06:07.507136 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.507142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.508391 (+ 1249us) server_negotiation.cc:407] Connection header received
0807 03:06:07.510278 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.510283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.510419 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.510507 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:07.511959 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.512794 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.514359 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.514786 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.514893 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.517976 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:07.517981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:07.517984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:07.518027 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:07.520472 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.521110 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.521115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.521117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.521243 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.521628 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.521633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.521636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.521900 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:07.522020 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.522717 (+ 697us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.522939 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260807 03:06:07.524219 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.507402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:35579)
0807 03:06:07.508229 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:07.508282 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:07.510027 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:07.510737 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:07.510745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:07.511218 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:07.511801 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.511818 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.512929 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.512932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.514198 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.514210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.514991 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.514998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.515214 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.515765 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:07.515788 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:07.517793 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:07.520592 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:07.520601 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:07.520614 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:07.521002 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:07.521355 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:07.521358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:07.521360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:07.521526 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:07.522028 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:07.522036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:07.522381 (+ 345us) client_negotiation.cc:769] Sending connection context
0807 03:06:07.522610 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 03:06:07.523075 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":347,"threads_started":1}
I20260807 03:06:07.525422 29353 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33843
I20260807 03:06:07.525663 29353 heartbeater.cc:464] Registering TS with master...
I20260807 03:06:07.526275 29353 heartbeater.cc:511] Master 127.5.122.62:33843 requested a full tablet report, sending...
I20260807 03:06:07.527554 28705 ts_manager.cc:194] Registered new tserver with Master: 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
I20260807 03:06:07.528299 28705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:35579
I20260807 03:06:07.533906 28981 heartbeater.cc:503] Master 127.5.122.62:33843 was elected leader, sending a full tablet report...
I20260807 03:06:07.536875 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:06:07.553437 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.541331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.5.122.62:33843)
0807 03:06:07.541521 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.541527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.541835 (+ 308us) server_negotiation.cc:407] Connection header received
0807 03:06:07.542002 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.542006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.542064 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.542142 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:07.543533 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.544273 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.545672 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.546077 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.546188 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.548464 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:07.548469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:07.548473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:07.548504 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:07.550901 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.551550 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.551555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.551558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.551687 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.552060 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.552064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.552066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.552369 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:07.552612 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.552913 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.553115 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:06:07.557194 28705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:06:07.563911 28705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:06:07.591975 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.577701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:34269 (local address 127.0.0.1:51932)
0807 03:06:07.584559 (+ 6858us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:07.584580 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:07.584726 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:07.585253 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:07.585258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:07.585289 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:07.585670 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.585681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.587558 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.587563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.589541 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.589553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.590961 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.590965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.591096 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.591131 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 03:06:07.591255 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 03:06:07.591452 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6704,"mutex_wait_us":185,"thread_start_us":341,"threads_started":1}
I20260807 03:06:07.592860 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.575704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:37601 (local address 127.0.0.1:36716)
0807 03:06:07.578201 (+ 2497us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:07.578225 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:07.578372 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:07.584431 (+ 6059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:07.584438 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:07.584479 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:07.584947 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.584960 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.587274 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.587279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.590258 (+ 2979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.590275 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.592054 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.592061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.592220 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.592251 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 03:06:07.592315 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 03:06:07.592433 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2380,"mutex_wait_us":251,"thread_start_us":1254,"threads_started":1}
I20260807 03:06:07.593070 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.578340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.5.122.3:34269)
0807 03:06:07.580182 (+ 1842us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.580192 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.584693 (+ 4501us) server_negotiation.cc:407] Connection header received
0807 03:06:07.584898 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.584904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.585071 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.585257 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:07.585852 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.587262 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.589688 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.590784 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.590926 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.591161 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.591355 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.592675 (+ 1320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1655,"thread_start_us":462,"threads_started":1}
I20260807 03:06:07.593739 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.576120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.5.122.2:37601)
0807 03:06:07.583777 (+ 7657us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.583786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.583814 (+ 28us) server_negotiation.cc:407] Connection header received
0807 03:06:07.583916 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.583924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.584181 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.585187 (+ 1006us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:07.585439 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.587122 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.590449 (+ 3327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.591818 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.593120 (+ 1302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.593312 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.593376 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.593509 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7500,"thread_start_us":7396,"threads_started":1}
I20260807 03:06:07.593761 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.575624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:46496)
0807 03:06:07.577311 (+ 1687us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:07.577355 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:07.577514 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:07.584746 (+ 7232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:07.584751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:07.584773 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:07.585220 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.585232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.587274 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.587279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.590264 (+ 2985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.590275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.592891 (+ 2616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.592898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.593083 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.593120 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 03:06:07.593223 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 03:06:07.593350 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1458,"mutex_wait_us":145,"thread_start_us":8630,"threads_started":1}
I20260807 03:06:07.594579 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.575862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.5.122.1:37953)
0807 03:06:07.584184 (+ 8322us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.584192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.584218 (+ 26us) server_negotiation.cc:407] Connection header received
0807 03:06:07.584325 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.584333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.584534 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.585428 (+ 894us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:07.585644 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.587122 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.591257 (+ 4135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.592626 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.592786 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.593012 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.594000 (+ 988us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.594161 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8169,"thread_start_us":8028,"threads_started":1}
I20260807 03:06:07.596901 29075 tablet_service.cc:1467] Processing CreateTablet for tablet 01da3efbe35b45e5bcbf39259de05c7c (DEFAULT_TABLE table=test-table [id=ef000199e9ae470a9ef6b499691cb35d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:06:07.596966 29266 tablet_service.cc:1467] Processing CreateTablet for tablet 01da3efbe35b45e5bcbf39259de05c7c (DEFAULT_TABLE table=test-table [id=ef000199e9ae470a9ef6b499691cb35d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:06:07.598228 29075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01da3efbe35b45e5bcbf39259de05c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:06:07.597880 28884 tablet_service.cc:1467] Processing CreateTablet for tablet 01da3efbe35b45e5bcbf39259de05c7c (DEFAULT_TABLE table=test-table [id=ef000199e9ae470a9ef6b499691cb35d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:06:07.598393 29266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01da3efbe35b45e5bcbf39259de05c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:06:07.599131 28884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01da3efbe35b45e5bcbf39259de05c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:06:07.610841 29378 tablet_bootstrap.cc:492] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Bootstrap starting.
I20260807 03:06:07.613427 29383 tablet_bootstrap.cc:492] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Bootstrap starting.
I20260807 03:06:07.614482 29378 tablet_bootstrap.cc:654] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Neither blocks nor log segments found. Creating new log.
I20260807 03:06:07.615134 29381 tablet_bootstrap.cc:492] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Bootstrap starting.
I20260807 03:06:07.615882 29378 log.cc:824] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:07.617959 29381 tablet_bootstrap.cc:654] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Neither blocks nor log segments found. Creating new log.
I20260807 03:06:07.619047 29381 log.cc:824] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:07.618069 29383 tablet_bootstrap.cc:654] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Neither blocks nor log segments found. Creating new log.
I20260807 03:06:07.620394 29383 log.cc:824] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:07.621980 29378 tablet_bootstrap.cc:492] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: No bootstrap required, opened a new log
I20260807 03:06:07.622041 29381 tablet_bootstrap.cc:492] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: No bootstrap required, opened a new log
I20260807 03:06:07.622225 29378 ts_tablet_manager.cc:1397] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260807 03:06:07.622395 29381 ts_tablet_manager.cc:1397] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260807 03:06:07.623413 29383 tablet_bootstrap.cc:492] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: No bootstrap required, opened a new log
I20260807 03:06:07.623683 29383 ts_tablet_manager.cc:1397] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.006s
I20260807 03:06:07.629673 29378 raft_consensus.cc:348] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.630020 29378 raft_consensus.cc:374] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:06:07.630127 29378 raft_consensus.cc:729] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a975be89644b7bb1cf3d9c41afac84, State: Initialized, Role: FOLLOWER
I20260807 03:06:07.630751 29378 consensus_queue.cc:260] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.631701 29165 heartbeater.cc:503] Master 127.5.122.62:33843 was elected leader, sending a full tablet report...
I20260807 03:06:07.633379 29378 ts_tablet_manager.cc:1428] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 03:06:07.633178 29381 raft_consensus.cc:348] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.633632 29381 raft_consensus.cc:374] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:06:07.633759 29381 raft_consensus.cc:729] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c33d52aab44873a803fc40e306d5a4, State: Initialized, Role: FOLLOWER
I20260807 03:06:07.634393 29381 consensus_queue.cc:260] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.635497 29353 heartbeater.cc:503] Master 127.5.122.62:33843 was elected leader, sending a full tablet report...
I20260807 03:06:07.636219 29381 ts_tablet_manager.cc:1428] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260807 03:06:07.635906 29383 raft_consensus.cc:348] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.636420 29383 raft_consensus.cc:374] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:06:07.636510 29383 raft_consensus.cc:729] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c3712a0d23410f8c6bfda2b8b2db96, State: Initialized, Role: FOLLOWER
I20260807 03:06:07.637120 29383 consensus_queue.cc:260] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.638901 29383 ts_tablet_manager.cc:1428] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 03:06:07.641534 29389 raft_consensus.cc:482] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:06:07.641732 29389 raft_consensus.cc:504] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.643174 29389 leader_election.cc:288] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953), 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601)
I20260807 03:06:07.649289 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.643520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:41569 (local address 127.5.122.1:37953)
0807 03:06:07.643690 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.643695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.644040 (+ 345us) server_negotiation.cc:407] Connection header received
0807 03:06:07.644220 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.644225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.644293 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.644386 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:07.644750 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.645584 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.647520 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.648377 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.648477 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.648570 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.648930 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.649061 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:06:07.649281 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.643331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.5.122.3:41569)
0807 03:06:07.643937 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:07.643960 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:07.644096 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:07.644426 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:07.644432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:07.644460 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:07.644620 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.644628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.645727 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.645734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.647366 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.647380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.648535 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.648542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.648761 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.648800 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 03:06:07.648891 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 03:06:07.649041 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":55}
I20260807 03:06:07.649698 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.643846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53105 (local address 127.5.122.2:37601)
0807 03:06:07.643997 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.644000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.644150 (+ 150us) server_negotiation.cc:407] Connection header received
0807 03:06:07.644298 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.644302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.644367 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.644453 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:07.645142 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.645954 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.647799 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.648748 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.648896 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.648994 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.649302 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.649435 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 03:06:07.649763 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.643482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:37601 (local address 127.5.122.3:53105)
0807 03:06:07.644072 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:07.644088 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:07.644196 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:07.644608 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:07.644611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:07.644632 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:07.644983 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.644995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.646100 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.646104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.647638 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:07.647652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.648955 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.648962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:07.649125 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.649163 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 03:06:07.649253 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 03:06:07.649375 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":20,"thread_start_us":704,"threads_started":1}
I20260807 03:06:07.650439 28914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01da3efbe35b45e5bcbf39259de05c7c" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" is_pre_election: true
I20260807 03:06:07.650883 29101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01da3efbe35b45e5bcbf39259de05c7c" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a975be89644b7bb1cf3d9c41afac84" is_pre_election: true
I20260807 03:06:07.650997 28914 raft_consensus.cc:2454] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 0.
I20260807 03:06:07.651381 29101 raft_consensus.cc:2454] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 0.
I20260807 03:06:07.651634 29198 leader_election.cc:302] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c33d52aab44873a803fc40e306d5a4, 76c3712a0d23410f8c6bfda2b8b2db96; no voters:
I20260807 03:06:07.651988 29389 raft_consensus.cc:2781] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:06:07.652128 29389 raft_consensus.cc:482] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:06:07.652225 29389 raft_consensus.cc:3037] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:06:07.654260 29389 raft_consensus.cc:504] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.654886 29389 leader_election.cc:288] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 election: Requested vote from peers 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953), 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601)
I20260807 03:06:07.655210 28914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01da3efbe35b45e5bcbf39259de05c7c" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c3712a0d23410f8c6bfda2b8b2db96"
I20260807 03:06:07.655402 28914 raft_consensus.cc:3037] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:06:07.655421 29101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01da3efbe35b45e5bcbf39259de05c7c" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a975be89644b7bb1cf3d9c41afac84"
I20260807 03:06:07.655611 29101 raft_consensus.cc:3037] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:06:07.657819 28914 raft_consensus.cc:2454] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 1.
I20260807 03:06:07.658131 29198 leader_election.cc:302] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c33d52aab44873a803fc40e306d5a4, 76c3712a0d23410f8c6bfda2b8b2db96; no voters:
I20260807 03:06:07.658380 29101 raft_consensus.cc:2454] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 1.
I20260807 03:06:07.658397 29389 raft_consensus.cc:2781] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:06:07.658891 29389 raft_consensus.cc:686] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 1 LEADER]: Becoming Leader. State: Replica: 40c33d52aab44873a803fc40e306d5a4, State: Running, Role: LEADER
I20260807 03:06:07.659503 29389 consensus_queue.cc:237] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:07.663277 28705 catalog_manager.cc:5495] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "40c33d52aab44873a803fc40e306d5a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } health_report { overall_health: HEALTHY } } }
I20260807 03:06:07.686383 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.681890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.5.122.3:34269)
0807 03:06:07.682045 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.682049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.682065 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:06:07.682205 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.682209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.682288 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.682445 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:06:07.682736 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.683425 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.684704 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.685068 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.685149 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.685520 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:06:07.685967 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:06:07.686065 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.686103 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.686161 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 03:06:07.693569 29101 raft_consensus.cc:1261] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Refusing update from remote peer 40c33d52aab44873a803fc40e306d5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:06:07.693567 28914 raft_consensus.cc:1261] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Refusing update from remote peer 40c33d52aab44873a803fc40e306d5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:06:07.694526 29396 consensus_queue.cc:1035] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:06:07.694854 29389 consensus_queue.cc:1035] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:06:07.704221 29402 mvcc.cc:205] Tried to move back new op lower bound from 7315750779663908864 to 7315750779536650240. Current Snapshot: MvccSnapshot[applied={T|T < 7315750779663908864}]
I20260807 03:06:07.705622 29403 mvcc.cc:205] Tried to move back new op lower bound from 7315750779663908864 to 7315750779536650240. Current Snapshot: MvccSnapshot[applied={T|T < 7315750779663908864}]
I20260807 03:06:07.721621 28705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:06:07.721894 28705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:06:07.725943 28705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Sending DeleteTablet for 3 replicas of tablet 01da3efbe35b45e5bcbf39259de05c7c
I20260807 03:06:07.726934 28884 tablet_service.cc:1514] Processing DeleteTablet for tablet 01da3efbe35b45e5bcbf39259de05c7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:06:07 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:46496
I20260807 03:06:07.727133 28883 tablet_service.cc:1514] Processing DeleteTablet for tablet 01da3efbe35b45e5bcbf39259de05c7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:06:07 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:46496
I20260807 03:06:07.727550 29075 tablet_service.cc:1514] Processing DeleteTablet for tablet 01da3efbe35b45e5bcbf39259de05c7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:06:07 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:36716
I20260807 03:06:07.727782 29417 tablet_replica.cc:330] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: stopping tablet replica
I20260807 03:06:07.727811 29266 tablet_service.cc:1514] Processing DeleteTablet for tablet 01da3efbe35b45e5bcbf39259de05c7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:06:07 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:51932
I20260807 03:06:07.728168 29417 raft_consensus.cc:2229] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:06:07.728724 29417 raft_consensus.cc:2258] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:06:07.728822 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 28795
I20260807 03:06:07.730865 29417 ts_tablet_manager.cc:1905] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:06:07.737866 29417 ts_tablet_manager.cc:1918] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:06:07.738126 29417 log.cc:1196] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/01da3efbe35b45e5bcbf39259de05c7c
I20260807 03:06:07.738734 29417 ts_tablet_manager.cc:1939] T 01da3efbe35b45e5bcbf39259de05c7c P 76c3712a0d23410f8c6bfda2b8b2db96: Deleting consensus metadata
I20260807 03:06:07.740593 28685 catalog_manager.cc:4841] TS 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953): tablet 01da3efbe35b45e5bcbf39259de05c7c (table test-table [id=ef000199e9ae470a9ef6b499691cb35d]) successfully deleted
W20260807 03:06:07.741631 28685 catalog_manager.cc:4808] TS 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953): delete failed for tablet 01da3efbe35b45e5bcbf39259de05c7c because the tablet was not found. No further retry: Not found: Tablet not found: 01da3efbe35b45e5bcbf39259de05c7c
I20260807 03:06:07.747627 29419 tablet_replica.cc:330] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: stopping tablet replica
I20260807 03:06:07.747879 29418 tablet_replica.cc:330] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: stopping tablet replica
I20260807 03:06:07.748217 29419 raft_consensus.cc:2229] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:06:07.748338 29418 raft_consensus.cc:2229] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:06:07.748872 29418 raft_consensus.cc:2258] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:06:07.748965 29419 raft_consensus.cc:2258] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:06:07.751647 29419 ts_tablet_manager.cc:1905] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:06:07.755164 29418 ts_tablet_manager.cc:1905] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 03:06:07 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:06:07.758620 29419 ts_tablet_manager.cc:1918] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:06:07.758821 29419 log.cc:1196] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/01da3efbe35b45e5bcbf39259de05c7c
I20260807 03:06:07.759438 29419 ts_tablet_manager.cc:1939] T 01da3efbe35b45e5bcbf39259de05c7c P 40c33d52aab44873a803fc40e306d5a4: Deleting consensus metadata
I20260807 03:06:07.761301 28684 catalog_manager.cc:4841] TS 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): tablet 01da3efbe35b45e5bcbf39259de05c7c (table test-table [id=ef000199e9ae470a9ef6b499691cb35d]) successfully deleted
I20260807 03:06:07.762207 29418 ts_tablet_manager.cc:1918] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:06:07.762385 29418 log.cc:1196] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/01da3efbe35b45e5bcbf39259de05c7c
I20260807 03:06:07.762885 29418 ts_tablet_manager.cc:1939] T 01da3efbe35b45e5bcbf39259de05c7c P 33a975be89644b7bb1cf3d9c41afac84: Deleting consensus metadata
I20260807 03:06:07.764662 28684 catalog_manager.cc:4841] TS 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): tablet 01da3efbe35b45e5bcbf39259de05c7c (table test-table [id=ef000199e9ae470a9ef6b499691cb35d]) successfully deleted
I20260807 03:06:07.765717 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.729966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46504 (local address 127.5.122.1:37953)
0807 03:06:07.730137 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.730142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.746945 (+ 16803us) server_negotiation.cc:407] Connection header received
0807 03:06:07.747164 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.747169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.747249 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.747335 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:07.749513 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.750601 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.752523 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.753141 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.753272 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.758977 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:07.758997 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:07.759014 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:07.759063 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:07.762868 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.763510 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.763518 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.763521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.763643 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.764027 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.764032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.764034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.764395 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:07.765016 (+ 621us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.765177 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.765393 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:06:07.766434 28975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:07.854296 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 28795
I20260807 03:06:07.880035 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 28988
Aug 07 03:06:07 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:06:07.897466 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:07.880842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.5.122.2:37601)
0807 03:06:07.881037 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:07.881043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:07.881129 (+ 86us) server_negotiation.cc:407] Connection header received
0807 03:06:07.881309 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:07.881315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:07.881398 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:07.881504 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:07.883086 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.884215 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.886198 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:07.886931 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:07.887071 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:07.891867 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:07.891888 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:07.891903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:07.891937 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:07.894911 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.895695 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.895707 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.895709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.895822 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:07.896171 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:07.896174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:07.896176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:07.896524 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:07.896731 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:07.897055 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 03:06:07.897210 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 03:06:07.898151 29158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:07.984584 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 28988
I20260807 03:06:08.004274 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 29171
Aug 07 03:06:08 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:06:08.020911 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:08.005136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.5.122.3:34269)
0807 03:06:08.005312 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:08.005317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:08.005337 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:06:08.005529 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:08.005533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:08.005606 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:08.005705 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:08.007145 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:08.008041 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:08.009408 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:08.009851 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:08.009962 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:08.014791 (+ 4829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:08.014805 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:08.014821 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:08.014855 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:08.017821 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:08.018714 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:08.018721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:08.018723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:08.018894 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:08.019322 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:08.019326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:08.019327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:08.019695 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:08.020013 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:08.020337 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 03:06:08.020563 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:06:08.021690 29347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:08.143206 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 29171
I20260807 03:06:08.167191 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 28658
I20260807 03:06:08.168241 28751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:08.275058 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 28658
I20260807 03:06:28.299304 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33843
--webserver_interface=127.5.122.62
--webserver_port=33687
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:06:28.633311 29429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:28.633567 29429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:28.633697 29429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:28.640806 29429 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 03:06:28.640890 29429 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 03:06:28.640951 29429 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 03:06:28.641022 29429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:06:28.641100 29429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:28.641187 29429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:28.641245 29429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:06:28.641326 29429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:06:28.650748 29429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=33687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:28.651849 29429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:28.653044 29429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:28.675442 29444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:28.675483 29440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:28.675508 29439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:28.675752 29429 server_base.cc:1034] running on GCE node
I20260807 03:06:28.676802 29429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:28.678212 29429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:28.679435 29429 hybrid_clock.cc:648] HybridClock initialized: now 1786071988679366 us; error 42 us; skew 500 ppm
I20260807 03:06:28.679781 29429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:28 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071988, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:28.686955 29429 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:06:28.688473 29429 webserver.cc:460] Webserver started at http://127.5.122.62:33687/ using document root <none> and password file <none>
I20260807 03:06:28.689203 29429 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:28.689328 29429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:28.695478 29429 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 03:06:28.698254 29455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:28.699589 29429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:06:28.699759 29429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "2acc297bdaa2448e86dd581d3e1f3b82"
format_stamp: "Formatted at 2026-08-07 03:06:05 on dist-test-slave-fgb3"
server_key: "b562e6e0ebd4992af1f95d70884637bb"
server_key_iv: "96a1979ea560696f60d7622ac0120ffd"
server_key_version: "encryptionkey@0"
I20260807 03:06:28.700734 29429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:28.711795 29429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:28.715830 29429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:28.716081 29429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:28.733418 29429 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:33843
I20260807 03:06:28.733456 29529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:33843 every 8 connection(s)
I20260807 03:06:28.734855 29429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:06:28.738245 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 29429
I20260807 03:06:28.739284 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:37953
--local_ip_for_outbound_sockets=127.5.122.1
--tserver_master_addrs=127.5.122.62:33843
--webserver_port=40521
--webserver_interface=127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:06:28.742705 29533 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:06:28.746803 29533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Bootstrap starting.
I20260807 03:06:28.766631 29533 log.cc:824] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:28.778239 29533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:06:28.778745 29533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Bootstrap complete.
I20260807 03:06:28.787441 29533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:28.788215 29533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acc297bdaa2448e86dd581d3e1f3b82, State: Initialized, Role: FOLLOWER
I20260807 03:06:28.788781 29533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:28.789003 29533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:06:28.789116 29533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:06:28.789276 29533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 1 FOLLOWER]: Advancing to term 2
I20260807 03:06:28.792994 29533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:28.793474 29533 leader_election.cc:302] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2acc297bdaa2448e86dd581d3e1f3b82; no voters:
I20260807 03:06:28.794062 29533 leader_election.cc:288] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 03:06:28.794199 29537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 03:06:28.795368 29537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 LEADER]: Becoming Leader. State: Replica: 2acc297bdaa2448e86dd581d3e1f3b82, State: Running, Role: LEADER
I20260807 03:06:28.795893 29537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:28.796268 29533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:06:28.799208 29540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2acc297bdaa2448e86dd581d3e1f3b82. Latest consensus state: current_term: 2 leader_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } } }
I20260807 03:06:28.799870 29540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: This master's current role is: LEADER
I20260807 03:06:28.799547 29538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } } }
I20260807 03:06:28.800413 29538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: This master's current role is: LEADER
I20260807 03:06:28.809499 29549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:06:28.818354 29549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:06:28.819788 29549 catalog_manager.cc:1259] Loaded cluster ID: 39be175e8dbd46dcac018ae0629ef998
I20260807 03:06:28.819937 29549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:06:28.823907 29549 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:06:28.825549 29549 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Loaded TSK: 0
I20260807 03:06:28.861955 29549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Generated new TSK 1
I20260807 03:06:28.862388 29549 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 03:06:29.083779 29535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:29.084033 29535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:29.084138 29535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:29.091885 29535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:29.091985 29535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:29.092128 29535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:06:29.101632 29535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:37953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=40521
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:29.102797 29535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:29.104086 29535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:29.118497 29573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:29.118534 29575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:29.119166 29572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:29.119568 29535 server_base.cc:1034] running on GCE node
I20260807 03:06:29.120340 29535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:29.121523 29535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:29.122783 29535 hybrid_clock.cc:648] HybridClock initialized: now 1786071989122682 us; error 70 us; skew 500 ppm
I20260807 03:06:29.123179 29535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:29 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:29.127287 29535 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:06:29.129062 29535 webserver.cc:460] Webserver started at http://127.5.122.1:40521/ using document root <none> and password file <none>
I20260807 03:06:29.129978 29535 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:29.130155 29535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:29.137082 29535 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:06:29.141891 29586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:29.143488 29535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:06:29.143675 29535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "76c3712a0d23410f8c6bfda2b8b2db96"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4a344f75afc414f35c40b605d4cc43ae"
server_key_iv: "1b75342e2220fe67583fb463ee299499"
server_key_version: "encryptionkey@0"
I20260807 03:06:29.145064 29535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:29.180095 29535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:29.185338 29535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:29.185668 29535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:29.187062 29535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:29.188858 29535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:29.188977 29535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:29.189086 29535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:29.189167 29535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:29.225018 29535 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:37953
I20260807 03:06:29.225061 29763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:37953 every 8 connection(s)
I20260807 03:06:29.226533 29535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:06:29.230849 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 29535
I20260807 03:06:29.231487 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:37601
--local_ip_for_outbound_sockets=127.5.122.2
--tserver_master_addrs=127.5.122.62:33843
--webserver_port=37909
--webserver_interface=127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:06:29 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:06:29.253420 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:29.229339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:46613 (local address 127.5.122.62:33843)
0807 03:06:29.230082 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:29.230094 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:29.230572 (+ 478us) server_negotiation.cc:407] Connection header received
0807 03:06:29.232766 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:29.232785 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:29.233235 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:29.233663 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:29.235011 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.236753 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.238249 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.239284 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.239493 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:29.245703 (+ 6210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:29.245712 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:29.245726 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:29.245769 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:29.248886 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:29.249822 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:29.249829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:29.249835 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:29.250026 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:29.250564 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:29.250568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:29.250570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:29.251062 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:29.251281 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:29.252102 (+ 821us) server_negotiation.cc:299] Negotiation successful
0807 03:06:29.252448 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":342,"threads_started":1}
I20260807 03:06:29.253605 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:29.229511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:46613)
0807 03:06:29.230386 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:29.230440 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:29.232201 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:29.233738 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:29.233745 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:29.234234 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:29.234802 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:29.234819 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.236907 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.236910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:29.238094 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:29.238106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.239875 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.239883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:29.240156 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:29.242523 (+ 2367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:29.242555 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:29.245385 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:29.249035 (+ 3650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:29.249043 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:29.249058 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:29.249655 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:29.250160 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:29.250165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:29.250168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:29.250448 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:29.251220 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:29.251229 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:29.251623 (+ 394us) client_negotiation.cc:769] Sending connection context
0807 03:06:29.251904 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 03:06:29.252424 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":358,"threads_started":1}
I20260807 03:06:29.256707 29764 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33843
I20260807 03:06:29.257018 29764 heartbeater.cc:464] Registering TS with master...
I20260807 03:06:29.257815 29764 heartbeater.cc:511] Master 127.5.122.62:33843 requested a full tablet report, sending...
I20260807 03:06:29.260336 29476 ts_manager.cc:194] Registered new tserver with Master: 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953)
I20260807 03:06:29.262544 29476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:46613
W20260807 03:06:29.570552 29770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:29.570819 29770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:29.570914 29770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:29.577840 29770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:29.577937 29770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:29.578071 29770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:06:29.587399 29770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:37601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=37909
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:29.588580 29770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:29.589818 29770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:29.602187 29776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:29.602295 29775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:29.602275 29778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:29.603260 29770 server_base.cc:1034] running on GCE node
I20260807 03:06:29.603847 29770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:29.604797 29770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:29.606004 29770 hybrid_clock.cc:648] HybridClock initialized: now 1786071989605966 us; error 31 us; skew 500 ppm
I20260807 03:06:29.606302 29770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:29 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:29.609359 29770 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:06:29.610829 29770 webserver.cc:460] Webserver started at http://127.5.122.2:37909/ using document root <none> and password file <none>
I20260807 03:06:29.611497 29770 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:29.611615 29770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:29.616909 29770 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 03:06:29.619597 29790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:29.620879 29770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:06:29.621027 29770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "33a975be89644b7bb1cf3d9c41afac84"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4f571a779c5e1acf5fb5e30266593133"
server_key_iv: "0cda1ccff40c344d0be1dab68f55d57a"
server_key_version: "encryptionkey@0"
I20260807 03:06:29.622002 29770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:29.634807 29770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:29.639057 29770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:29.639323 29770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:29.640336 29770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:29.641767 29770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:29.641870 29770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:29.641974 29770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:29.642042 29770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:29.672477 29770 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:37601
I20260807 03:06:29.672538 29961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:37601 every 8 connection(s)
I20260807 03:06:29.673880 29770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:06:29.675097 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 29770
I20260807 03:06:29.675879 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:34269
--local_ip_for_outbound_sockets=127.5.122.3
--tserver_master_addrs=127.5.122.62:33843
--webserver_port=42799
--webserver_interface=127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 03:06:29 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:06:29.694988 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:29.676581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:47741 (local address 127.5.122.62:33843)
0807 03:06:29.676787 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:29.676793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:29.678034 (+ 1241us) server_negotiation.cc:407] Connection header received
0807 03:06:29.680283 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:29.680287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:29.680362 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:29.680446 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:29.682234 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.682954 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.684944 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.685331 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.685427 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:29.689248 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:29.689253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:29.689257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:29.689294 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:29.692174 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:29.692853 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:29.692858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:29.692861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:29.692998 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:29.693365 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:29.693369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:29.693373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:29.693708 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:29.693845 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:29.694452 (+ 607us) server_negotiation.cc:299] Negotiation successful
0807 03:06:29.694669 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 03:06:29.695726 29967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:29.677065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:47741)
0807 03:06:29.677851 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:29.677902 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:29.680019 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:29.680688 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:29.680699 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:29.681361 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:29.682088 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:29.682112 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.683111 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.683116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:29.684820 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:29.684835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:29.685547 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:29.685554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:29.685813 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:29.686503 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:29.686529 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:29.689018 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:29.692308 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:29.692315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:29.692347 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:29.692737 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:29.693092 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:29.693095 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:29.693097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:29.693262 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:29.693806 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:29.693811 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:29.694084 (+ 273us) client_negotiation.cc:769] Sending connection context
0807 03:06:29.694297 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 03:06:29.694706 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":301,"threads_started":1}
I20260807 03:06:29.697078 29964 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33843
I20260807 03:06:29.697363 29964 heartbeater.cc:464] Registering TS with master...
I20260807 03:06:29.698050 29964 heartbeater.cc:511] Master 127.5.122.62:33843 requested a full tablet report, sending...
I20260807 03:06:29.699477 29476 ts_manager.cc:194] Registered new tserver with Master: 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601)
I20260807 03:06:29.700253 29476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:47741
W20260807 03:06:30.003654 29968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:30.003952 29968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:30.004110 29968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:30.011552 29968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:30.011675 29968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:30.011832 29968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:06:30.021433 29968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:34269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=42799
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:30.022712 29968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:30.024015 29968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:30.036469 29977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:30.036511 29980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:30.036543 29978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:30.037312 29968 server_base.cc:1034] running on GCE node
I20260807 03:06:30.037837 29968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:30.038822 29968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:30.040021 29968 hybrid_clock.cc:648] HybridClock initialized: now 1786071990039978 us; error 37 us; skew 500 ppm
I20260807 03:06:30.040316 29968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:30 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:30.043354 29968 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:06:30.044824 29968 webserver.cc:460] Webserver started at http://127.5.122.3:42799/ using document root <none> and password file <none>
I20260807 03:06:30.045464 29968 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:30.045581 29968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:30.050974 29968 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:06:30.053619 29990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:30.054894 29968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:06:30.055037 29968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "40c33d52aab44873a803fc40e306d5a4"
format_stamp: "Formatted at 2026-08-07 03:06:07 on dist-test-slave-fgb3"
server_key: "698549b05a7d897a13d1806f7ea1c965"
server_key_iv: "78ff8ba02484bcbd6a12d1f64a528281"
server_key_version: "encryptionkey@0"
I20260807 03:06:30.055967 29968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:30.083017 29968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:30.086827 29968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:30.087107 29968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:30.088078 29968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:30.089404 29968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:30.089500 29968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:30.089607 29968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:30.089674 29968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:30.120795 29968 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:34269
I20260807 03:06:30.120877 30174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:34269 every 8 connection(s)
I20260807 03:06:30.122179 29968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:06:30.127696 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 29968
Aug 07 03:06:30 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786071990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:06:30.143177 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.124610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:41433 (local address 127.5.122.62:33843)
0807 03:06:30.124785 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.124790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.125868 (+ 1078us) server_negotiation.cc:407] Connection header received
0807 03:06:30.127876 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.127883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.127956 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.128042 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:30.129709 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.130831 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.132708 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.133295 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.133420 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.137272 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:30.137276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:30.137280 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:30.137310 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:30.140269 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.140958 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.140962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.140964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.141074 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.141488 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.141493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.141495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.141758 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:30.141898 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.142579 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.142790 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:06:30.143857 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.128636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.5.122.62:33843)
0807 03:06:30.129117 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.129123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.129476 (+ 353us) server_negotiation.cc:407] Connection header received
0807 03:06:30.129605 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.129610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.129684 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.129771 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:30.131681 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.132756 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.134490 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.135119 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.135247 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.138243 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:30.138248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:30.138251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:30.138301 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:30.140947 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.141558 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.141563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.141567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.141678 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.142033 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.142038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.142041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.142302 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:30.142571 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.142877 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.143078 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":249,"threads_started":1}
I20260807 03:06:30.144315 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.125002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:41433)
0807 03:06:30.125698 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.125735 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:30.127562 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:30.128300 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:30.128309 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:30.128893 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:30.129541 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.129564 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.130996 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.131000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.132570 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.132584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.133501 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.133505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.133699 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.134390 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:30.134415 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:30.137072 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:30.140405 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:30.140413 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:30.140429 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:30.140845 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:30.141178 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:30.141183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:30.141187 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:30.141383 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:30.141889 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:30.141897 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:30.142206 (+ 309us) client_negotiation.cc:769] Sending connection context
0807 03:06:30.142484 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 03:06:30.142992 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":274,"threads_started":1}
I20260807 03:06:30.145658 30175 heartbeater.cc:347] Connected to a master server at 127.5.122.62:33843
I20260807 03:06:30.145988 30175 heartbeater.cc:464] Registering TS with master...
I20260807 03:06:30.146854 30175 heartbeater.cc:511] Master 127.5.122.62:33843 requested a full tablet report, sending...
I20260807 03:06:30.148137 29476 ts_manager.cc:194] Registered new tserver with Master: 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
I20260807 03:06:30.148901 29476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:41433
I20260807 03:06:30.158594 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:06:30.165120 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.159513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.5.122.62:33843)
0807 03:06:30.159693 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.159697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.160190 (+ 493us) server_negotiation.cc:407] Connection header received
0807 03:06:30.160385 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.160389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.160526 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.160687 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:06:30.161098 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.161792 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.163293 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.163737 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.163853 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.164318 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:06:30.164605 (+ 287us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 03:06:30.164909 (+ 304us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58188: token expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 03:06:30.165256 30179 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58188: token expired
W20260807 03:06:30.165736 29359 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.122.62:33843)
I20260807 03:06:30.177774 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.166157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.5.122.62:33843)
0807 03:06:30.166334 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.166342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.166363 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:06:30.166411 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.166414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.166468 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.166549 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:30.167815 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.168544 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.169791 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.170196 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.170305 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.172766 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:30.172771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:30.172774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:30.172807 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:30.175416 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.176083 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.176087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.176090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.176228 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.176610 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.176612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.176614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.176924 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:30.177124 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.177350 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.177509 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 03:06:30.181099 29476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:06:30.188210 29476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:06:30.224514 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.199263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:53934)
0807 03:06:30.209815 (+ 10552us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.209860 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:30.211196 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:30.216635 (+ 5439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:30.216644 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:30.216682 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:30.217291 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.217307 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.219027 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.219032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.221851 (+ 2819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.221871 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.223523 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.223529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.223709 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.223750 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 03:06:30.223865 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 03:06:30.224049 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10288,"mutex_wait_us":40,"thread_start_us":17853,"threads_started":1}
I20260807 03:06:30.224444 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.199337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.5.122.1:37953)
0807 03:06:30.215915 (+ 16578us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.215923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.215953 (+ 30us) server_negotiation.cc:407] Connection header received
0807 03:06:30.216062 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.216071 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.216375 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.216582 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:30.217529 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.218855 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.222044 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.223318 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.223440 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.223645 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.223927 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.224096 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16386,"thread_start_us":16277,"threads_started":1}
I20260807 03:06:30.225297 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.210261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:34269 (local address 127.0.0.1:58992)
0807 03:06:30.217403 (+ 7142us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.217418 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:30.217516 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:30.218709 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:30.218714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:30.218739 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:30.219143 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.219154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.220483 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.220489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.222726 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.222743 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.224550 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.224557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.224718 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.224751 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:06:30.224814 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 03:06:30.224941 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7023,"mutex_wait_us":41,"thread_start_us":5436,"threads_started":1}
I20260807 03:06:30.226241 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.210642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.5.122.3:34269)
0807 03:06:30.218126 (+ 7484us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.218135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.218168 (+ 33us) server_negotiation.cc:407] Connection header received
0807 03:06:30.218258 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.218265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.218522 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.218707 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:30.219314 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.220331 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.222907 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.224285 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.225177 (+ 892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.225668 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.225738 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.225909 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7293,"thread_start_us":7108,"threads_started":1}
I20260807 03:06:30.227393 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.199308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:37601 (local address 127.0.0.1:33002)
0807 03:06:30.216542 (+ 17234us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.216560 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:30.216669 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:30.217888 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:30.217892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:30.217917 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:30.218281 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.218294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.220417 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.220423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.222419 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.222432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.226657 (+ 4225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.226663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.226862 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.226894 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:06:30.226961 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 03:06:30.227079 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17114,"mutex_wait_us":43,"thread_start_us":10077,"threads_started":1}
I20260807 03:06:30.228138 30191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.199889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.5.122.2:37601)
0807 03:06:30.215478 (+ 15589us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.215487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.217332 (+ 1845us) server_negotiation.cc:407] Connection header received
0807 03:06:30.217440 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.217447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.217661 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.218530 (+ 869us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:30.218762 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.220267 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.222565 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.223937 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.224060 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.227727 (+ 3667us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.227777 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.227908 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15395,"thread_start_us":16452,"threads_started":1}
I20260807 03:06:30.229332 29662 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3070c957504c43981987b64b5258ff (DEFAULT_TABLE table=test-table [id=ceee940badb649909b34c4b508b87b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:06:30.229743 30068 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3070c957504c43981987b64b5258ff (DEFAULT_TABLE table=test-table [id=ceee940badb649909b34c4b508b87b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:06:30.230984 29662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3070c957504c43981987b64b5258ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:06:30.231124 30068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3070c957504c43981987b64b5258ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:06:30.233961 29866 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3070c957504c43981987b64b5258ff (DEFAULT_TABLE table=test-table [id=ceee940badb649909b34c4b508b87b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:06:30.235131 29866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3070c957504c43981987b64b5258ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:06:30.245994 30197 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Bootstrap starting.
I20260807 03:06:30.247934 30199 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Bootstrap starting.
I20260807 03:06:30.251199 30199 tablet_bootstrap.cc:654] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Neither blocks nor log segments found. Creating new log.
I20260807 03:06:30.251199 30197 tablet_bootstrap.cc:654] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Neither blocks nor log segments found. Creating new log.
I20260807 03:06:30.251250 30200 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Bootstrap starting.
I20260807 03:06:30.252497 30199 log.cc:824] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:30.252497 30197 log.cc:824] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:30.254132 30200 tablet_bootstrap.cc:654] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Neither blocks nor log segments found. Creating new log.
I20260807 03:06:30.255249 30197 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: No bootstrap required, opened a new log
I20260807 03:06:30.255353 30200 log.cc:824] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:30.255486 30197 ts_tablet_manager.cc:1397] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260807 03:06:30.255614 30199 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: No bootstrap required, opened a new log
I20260807 03:06:30.255887 30199 ts_tablet_manager.cc:1397] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 03:06:30.258349 30200 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: No bootstrap required, opened a new log
I20260807 03:06:30.258594 30200 ts_tablet_manager.cc:1397] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260807 03:06:30.263382 30197 raft_consensus.cc:348] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.263712 30197 raft_consensus.cc:374] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:06:30.263811 30197 raft_consensus.cc:729] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a975be89644b7bb1cf3d9c41afac84, State: Initialized, Role: FOLLOWER
I20260807 03:06:30.264346 30197 consensus_queue.cc:260] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.265341 29964 heartbeater.cc:503] Master 127.5.122.62:33843 was elected leader, sending a full tablet report...
I20260807 03:06:30.265772 29764 heartbeater.cc:503] Master 127.5.122.62:33843 was elected leader, sending a full tablet report...
I20260807 03:06:30.266216 30197 ts_tablet_manager.cc:1428] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260807 03:06:30.265766 30199 raft_consensus.cc:348] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.266692 30199 raft_consensus.cc:374] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:06:30.266793 30199 raft_consensus.cc:729] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c3712a0d23410f8c6bfda2b8b2db96, State: Initialized, Role: FOLLOWER
I20260807 03:06:30.267379 30199 consensus_queue.cc:260] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.268648 30200 raft_consensus.cc:348] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.269066 30200 raft_consensus.cc:374] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:06:30.269170 30200 raft_consensus.cc:729] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c33d52aab44873a803fc40e306d5a4, State: Initialized, Role: FOLLOWER
I20260807 03:06:30.269732 30200 consensus_queue.cc:260] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.270885 30175 heartbeater.cc:503] Master 127.5.122.62:33843 was elected leader, sending a full tablet report...
I20260807 03:06:30.271539 30200 ts_tablet_manager.cc:1428] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260807 03:06:30.273722 30199 ts_tablet_manager.cc:1428] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Time spent starting tablet: real 0.018s user 0.013s sys 0.003s
W20260807 03:06:30.373752 30176 tablet.cc:2369] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:06:30.425889 29966 tablet.cc:2369] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:06:30.438243 30209 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:06:30.438582 30209 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.440231 30209 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953), 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601)
I20260807 03:06:30.446215 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.440678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:54397 (local address 127.5.122.1:37953)
0807 03:06:30.440895 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.440901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.440918 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:06:30.440993 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.440998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.441073 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.441155 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:30.441710 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.442657 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.444392 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.445408 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.445508 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.445575 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.445897 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.445979 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 03:06:30.446295 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.440427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.5.122.3:54397)
0807 03:06:30.440646 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.440663 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:30.440794 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:30.441325 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:30.441330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:30.441380 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:30.441558 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.441570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.442791 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.442797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.444239 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.444252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.445559 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.445566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.445750 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.445785 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 03:06:30.445884 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 03:06:30.446014 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 03:06:30.446934 30191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.441139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:50909 (local address 127.5.122.2:37601)
0807 03:06:30.441333 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.441338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.441463 (+ 125us) server_negotiation.cc:407] Connection header received
0807 03:06:30.441612 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.441617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.441694 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.441764 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:06:30.442351 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.443218 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.445034 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.445987 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.446107 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.446197 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.446566 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.446687 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 03:06:30.446935 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.440637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:37601 (local address 127.5.122.3:50909)
0807 03:06:30.441374 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.441389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:30.441496 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:30.441805 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:30.441810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:30.441835 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:06:30.442183 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.442194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.443348 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.443352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.444887 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:30.444901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.446187 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.446194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:30.446395 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.446427 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:06:30.446512 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 03:06:30.446627 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":406,"threads_started":1}
I20260807 03:06:30.447293 29690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3070c957504c43981987b64b5258ff" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" is_pre_election: true
I20260807 03:06:30.447805 29690 raft_consensus.cc:2454] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 0.
I20260807 03:06:30.447975 29899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3070c957504c43981987b64b5258ff" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a975be89644b7bb1cf3d9c41afac84" is_pre_election: true
I20260807 03:06:30.448452 29899 raft_consensus.cc:2454] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 0.
I20260807 03:06:30.448509 29993 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c33d52aab44873a803fc40e306d5a4, 76c3712a0d23410f8c6bfda2b8b2db96; no voters:
I20260807 03:06:30.448877 30209 raft_consensus.cc:2781] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:06:30.449033 30209 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:06:30.449141 30209 raft_consensus.cc:3037] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:06:30.451375 30209 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.451908 30209 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 election: Requested vote from peers 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953), 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601)
I20260807 03:06:30.452325 29690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3070c957504c43981987b64b5258ff" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c3712a0d23410f8c6bfda2b8b2db96"
I20260807 03:06:30.452394 29899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3070c957504c43981987b64b5258ff" candidate_uuid: "40c33d52aab44873a803fc40e306d5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a975be89644b7bb1cf3d9c41afac84"
I20260807 03:06:30.452533 29690 raft_consensus.cc:3037] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:06:30.452577 29899 raft_consensus.cc:3037] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:06:30.455665 29690 raft_consensus.cc:2454] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 1.
I20260807 03:06:30.455665 29899 raft_consensus.cc:2454] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c33d52aab44873a803fc40e306d5a4 in term 1.
I20260807 03:06:30.456054 29993 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c33d52aab44873a803fc40e306d5a4, 76c3712a0d23410f8c6bfda2b8b2db96; no voters:
I20260807 03:06:30.456326 30209 raft_consensus.cc:2781] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:06:30.456763 30209 raft_consensus.cc:686] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 1 LEADER]: Becoming Leader. State: Replica: 40c33d52aab44873a803fc40e306d5a4, State: Running, Role: LEADER
I20260807 03:06:30.457243 30209 consensus_queue.cc:237] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:30.461669 29476 catalog_manager.cc:5495] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "40c33d52aab44873a803fc40e306d5a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } health_report { overall_health: HEALTHY } } }
W20260807 03:06:30.478772 29766 tablet.cc:2369] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:06:30.500962 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.495585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59004 (local address 127.5.122.3:34269)
0807 03:06:30.495768 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.495772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.495789 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:06:30.496049 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.496053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.496136 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.496265 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:06:30.496662 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.497399 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.498896 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.499293 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.499406 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.499906 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:06:30.500387 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:06:30.500519 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.500620 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.500699 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:06:30.509052 29690 raft_consensus.cc:1261] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Refusing update from remote peer 40c33d52aab44873a803fc40e306d5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:06:30.509052 29899 raft_consensus.cc:1261] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Refusing update from remote peer 40c33d52aab44873a803fc40e306d5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:06:30.510100 30211 consensus_queue.cc:1035] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:06:30.510488 30209 consensus_queue.cc:1035] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:06:30.526499 30220 mvcc.cc:205] Tried to move back new op lower bound from 7315750873114554368 to 7315750872916336640. Current Snapshot: MvccSnapshot[applied={T|T < 7315750873114554368}]
I20260807 03:06:30.534669 29475 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:06:30.534934 29475 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:06:30.538283 29475 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Sending DeleteTablet for 3 replicas of tablet 2e3070c957504c43981987b64b5258ff
I20260807 03:06:30.539484 29662 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3070c957504c43981987b64b5258ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:06:30 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:53934
I20260807 03:06:30.539593 29866 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3070c957504c43981987b64b5258ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:06:30 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:33002
I20260807 03:06:30.540076 30068 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3070c957504c43981987b64b5258ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:06:30 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:58992
I20260807 03:06:30.540292 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 29535
I20260807 03:06:30.549597 30236 tablet_replica.cc:330] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: stopping tablet replica
I20260807 03:06:30.549775 30235 tablet_replica.cc:330] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: stopping tablet replica
I20260807 03:06:30.550179 30235 raft_consensus.cc:2229] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:06:30.550268 30236 raft_consensus.cc:2229] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:06:30.550711 30235 raft_consensus.cc:2258] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:06:30.551174 30236 raft_consensus.cc:2258] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:06:30.553194 30235 ts_tablet_manager.cc:1905] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:06:30.553231 30236 ts_tablet_manager.cc:1905] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:06:30.560256 30235 ts_tablet_manager.cc:1918] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:06:30.560436 30235 log.cc:1196] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/2e3070c957504c43981987b64b5258ff
I20260807 03:06:30.560563 30236 ts_tablet_manager.cc:1918] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:06:30.560789 30236 log.cc:1196] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/2e3070c957504c43981987b64b5258ff
I20260807 03:06:30.561017 30235 ts_tablet_manager.cc:1939] T 2e3070c957504c43981987b64b5258ff P 33a975be89644b7bb1cf3d9c41afac84: Deleting consensus metadata
I20260807 03:06:30.561407 30236 ts_tablet_manager.cc:1939] T 2e3070c957504c43981987b64b5258ff P 40c33d52aab44873a803fc40e306d5a4: Deleting consensus metadata
I20260807 03:06:30.563113 29456 catalog_manager.cc:4841] TS 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): tablet 2e3070c957504c43981987b64b5258ff (table test-table [id=ceee940badb649909b34c4b508b87b1f]) successfully deleted
I20260807 03:06:30.563409 29456 catalog_manager.cc:4841] TS 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): tablet 2e3070c957504c43981987b64b5258ff (table test-table [id=ceee940badb649909b34c4b508b87b1f]) successfully deleted
I20260807 03:06:30.564302 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.541768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53950 (local address 127.5.122.1:37953)
0807 03:06:30.541946 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.541952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.541969 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:06:30.542020 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.542025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.542120 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.542213 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:30.544109 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.545221 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.547181 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.547801 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.547932 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.555264 (+ 7332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:30.555281 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:30.555297 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:30.555338 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:30.559298 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.561009 (+ 1711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.561019 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.561023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.561195 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.562139 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.562144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.562147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.562670 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:30.562966 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.563857 (+ 891us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.564039 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:06:30.564950 29757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:30.653506 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 29535
I20260807 03:06:30.682564 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 29770
W20260807 03:06:30.682693 29457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.1:37953 (error 108)
I20260807 03:06:30.695981 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.695558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:53966)
0807 03:06:30.695737 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.695837 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 03:06:30.696439 29457 catalog_manager.cc:4584] TS 76c3712a0d23410f8c6bfda2b8b2db96 (127.5.122.1:37953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e3070c957504c43981987b64b5258ff: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
I20260807 03:06:30.700992 30191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.684170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.5.122.2:37601)
0807 03:06:30.684349 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.684355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.684373 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:06:30.684447 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.684452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.684552 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.684668 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:30.686586 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.687773 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.689688 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.690357 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.690487 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.694037 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:30.694053 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:30.694071 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:30.694113 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:30.698262 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.698969 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.698978 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.698982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.699137 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.699447 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.699452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.699455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.699929 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:30.700233 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.700418 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.700645 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:06:30.701673 29957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:30.750401 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.749908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:53970)
0807 03:06:30.750093 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.750192 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 03:06:30.751721 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.751339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:53986)
0807 03:06:30.751478 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.751569 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 03:06:30.804211 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 29770
I20260807 03:06:30.812589 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.812027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:53994)
0807 03:06:30.812267 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.812388 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 03:06:30.813843 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.813410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:54006)
0807 03:06:30.813586 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.813682 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 03:06:30.825604 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 29968
I20260807 03:06:30.841637 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.826345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.5.122.3:34269)
0807 03:06:30.826493 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:30.826498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:30.826530 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:06:30.826709 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:30.826713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:30.826776 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:30.826875 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:30.828712 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.829486 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.831409 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:30.831841 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:30.831935 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:30.835458 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:30.835471 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:30.835489 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:30.835526 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:30.838650 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.839516 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.839523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.839525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.839673 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:30.840142 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:30.840146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:30.840148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:30.840556 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:30.840809 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:30.841156 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 03:06:30.841315 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 03:06:30.842399 30170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:30.894513 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.893859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:54020)
0807 03:06:30.894118 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.894249 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 03:06:30.895706 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:30.895267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:37953 (local address 127.0.0.1:54028)
0807 03:06:30.895484 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:30.895572 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 03:06:30.963131 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 29968
I20260807 03:06:30.988516 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 29429
I20260807 03:06:30.989637 29521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:06:31.101722 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 29429
I20260807 03:06:51.127753 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33843
--webserver_interface=127.5.122.62
--webserver_port=33687
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:06:51.460973 30244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:51.461252 30244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:51.461387 30244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:51.468604 30244 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 03:06:51.468708 30244 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 03:06:51.468768 30244 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 03:06:51.468849 30244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:06:51.468922 30244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:51.469018 30244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:51.469082 30244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:06:51.469157 30244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:06:51.479250 30244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=33687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:51.480583 30244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:51.481976 30244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:51.493479 30253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:51.493556 30258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:51.493459 30255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:51.494617 30244 server_base.cc:1034] running on GCE node
I20260807 03:06:51.495476 30244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:51.496943 30244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:51.498200 30244 hybrid_clock.cc:648] HybridClock initialized: now 1786072011498136 us; error 35 us; skew 500 ppm
I20260807 03:06:51.498571 30244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:51 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:51.505113 30244 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:06:51.506567 30244 webserver.cc:460] Webserver started at http://127.5.122.62:33687/ using document root <none> and password file <none>
I20260807 03:06:51.507229 30244 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:51.507342 30244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:51.512622 30244 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 03:06:51.515285 30267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:51.516469 30244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:06:51.516619 30244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "2acc297bdaa2448e86dd581d3e1f3b82"
format_stamp: "Formatted at 2026-08-07 03:06:05 on dist-test-slave-fgb3"
server_key: "b562e6e0ebd4992af1f95d70884637bb"
server_key_iv: "96a1979ea560696f60d7622ac0120ffd"
server_key_version: "encryptionkey@0"
I20260807 03:06:51.517530 30244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:51.538714 30244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:51.543155 30244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:51.543448 30244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:51.561379 30244 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:33843
I20260807 03:06:51.561419 30343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:33843 every 8 connection(s)
I20260807 03:06:51.562906 30244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:06:51.569095 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 30244
I20260807 03:06:51.570169 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:37953
--local_ip_for_outbound_sockets=127.5.122.1
--tserver_master_addrs=127.5.122.62:33843
--webserver_port=40521
--webserver_interface=127.5.122.1
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:06:51.570331 30345 sys_catalog.cc:263] Verifying existing consensus state
I20260807 03:06:51.574826 30345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Bootstrap starting.
I20260807 03:06:51.596873 30345 log.cc:824] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:51.612844 30345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:06:51.613384 30345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Bootstrap complete.
I20260807 03:06:51.622596 30345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:51.623476 30345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acc297bdaa2448e86dd581d3e1f3b82, State: Initialized, Role: FOLLOWER
I20260807 03:06:51.624074 30345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:51.624294 30345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:06:51.624404 30345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:06:51.624550 30345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 2 FOLLOWER]: Advancing to term 3
I20260807 03:06:51.628543 30345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:51.629024 30345 leader_election.cc:302] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2acc297bdaa2448e86dd581d3e1f3b82; no voters:
I20260807 03:06:51.629630 30345 leader_election.cc:288] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 03:06:51.629768 30351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 03:06:51.630867 30351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [term 3 LEADER]: Becoming Leader. State: Replica: 2acc297bdaa2448e86dd581d3e1f3b82, State: Running, Role: LEADER
I20260807 03:06:51.631430 30351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } }
I20260807 03:06:51.631820 30345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:06:51.633816 30353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2acc297bdaa2448e86dd581d3e1f3b82. Latest consensus state: current_term: 3 leader_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } } }
I20260807 03:06:51.634146 30353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: This master's current role is: LEADER
I20260807 03:06:51.634689 30352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acc297bdaa2448e86dd581d3e1f3b82" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33843 } } }
I20260807 03:06:51.635016 30352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82 [sys.catalog]: This master's current role is: LEADER
I20260807 03:06:51.641060 30359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:06:51.649386 30359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:06:51.650817 30359 catalog_manager.cc:1259] Loaded cluster ID: 39be175e8dbd46dcac018ae0629ef998
I20260807 03:06:51.651188 30359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:06:51.655259 30359 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:06:51.656826 30359 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Loaded TSK: 0
I20260807 03:06:51.657033 30359 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2acc297bdaa2448e86dd581d3e1f3b82: Loaded TSK: 1
F20260807 03:06:51.658145 30359 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc69f4979ad google::LogMessage::Fail() at ??:0
@ 0x7fc69f49a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc69f4974de google::LogMessage::Flush() at ??:0
@ 0x7fc69f498a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc6a31fb7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc6a31ea73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc6b4ec61c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc6b4ec5783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc6b4f512a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fc6b4f510c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc6b4f9b5e9 std::function<>::operator()() at ??:0
@ 0x7fc6b4ec9cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc6b4ec7c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc6b4f4f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc6b4f4f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc6a20e3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc6a20e691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc6a20e6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc6a20c1eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc6a539b6db start_thread at ??:0
@ 0x7fc69e33161f clone at ??:0
W20260807 03:06:51.952047 30349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:51.952335 30349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:51.952478 30349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:51.959692 30349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:51.959820 30349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:51.959980 30349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:06:51.969726 30349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:37953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=40521
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:51.971041 30349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:51.972359 30349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:51.985149 30418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:51.985073 30419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:51.985106 30423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:51.985970 30349 server_base.cc:1034] running on GCE node
I20260807 03:06:51.986603 30349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:51.987505 30349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:51.988698 30349 hybrid_clock.cc:648] HybridClock initialized: now 1786072011988650 us; error 37 us; skew 500 ppm
I20260807 03:06:51.989004 30349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:51 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:51.996410 30349 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:06:51.997881 30349 webserver.cc:460] Webserver started at http://127.5.122.1:40521/ using document root <none> and password file <none>
I20260807 03:06:51.998656 30349 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:51.998778 30349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:52.004199 30349 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:06:52.007035 30432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:52.008611 30349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:06:52.008782 30349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "76c3712a0d23410f8c6bfda2b8b2db96"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4a344f75afc414f35c40b605d4cc43ae"
server_key_iv: "1b75342e2220fe67583fb463ee299499"
server_key_version: "encryptionkey@0"
I20260807 03:06:52.010135 30349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:52.029657 30349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:52.038374 30349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:52.038656 30349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:52.039907 30349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:52.042361 30442 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 03:06:52.048357 30349 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 03:06:52.048521 30349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260807 03:06:52.048655 30349 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 03:06:52.053367 30349 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 03:06:52.053481 30349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s user 0.002s sys 0.000s
I20260807 03:06:52.053824 30442 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Bootstrap starting.
I20260807 03:06:52.080766 30442 log.cc:824] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Log is configured to *not* fsync() on all Append() calls
I20260807 03:06:52.090435 30349 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:37953
I20260807 03:06:52.090507 30596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:37953 every 8 connection(s)
I20260807 03:06:52.091622 30349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:06:52.093633 30442 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 03:06:52.094297 30442 tablet_bootstrap.cc:492] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Bootstrap complete.
I20260807 03:06:52.095953 30442 ts_tablet_manager.cc:1397] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Time spent bootstrapping tablet: real 0.042s user 0.032s sys 0.008s
I20260807 03:06:52.096451 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.094508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:36677)
0807 03:06:52.095345 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.095567 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":371,"threads_started":1}
W20260807 03:06:52.097054 30597 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:33843 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.62:33843: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
I20260807 03:06:52.097908 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.097574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:37961)
0807 03:06:52.097723 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.097796 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 03:06:52.098814 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.098461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:41427)
0807 03:06:52.098598 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.098684 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 03:06:52.099040 30597 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 03:06:52.101902 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 30349
I20260807 03:06:52.102741 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:37601
--local_ip_for_outbound_sockets=127.5.122.2
--tserver_master_addrs=127.5.122.62:33843
--webserver_port=37909
--webserver_interface=127.5.122.2
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:06:52.102914 30442 raft_consensus.cc:348] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:52.104008 30442 raft_consensus.cc:729] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c3712a0d23410f8c6bfda2b8b2db96, State: Initialized, Role: FOLLOWER
I20260807 03:06:52.104614 30442 consensus_queue.cc:260] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:52.105999 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.105689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:45111)
0807 03:06:52.105822 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.105888 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 03:06:52.106079 30442 ts_tablet_manager.cc:1428] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260807 03:06:52.106914 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.106631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:50343)
0807 03:06:52.106748 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.106808 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 03:06:52.427505 30604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:52.427794 30604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:52.427927 30604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:52.435030 30604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:52.435154 30604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:52.435312 30604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:06:52.445253 30604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:37601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=37909
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:52.446642 30604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:52.447995 30604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:52.461023 30610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:52.460901 30611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:52.460871 30613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:52.462016 30604 server_base.cc:1034] running on GCE node
I20260807 03:06:52.462709 30604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:52.463665 30604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:52.464871 30604 hybrid_clock.cc:648] HybridClock initialized: now 1786072012464820 us; error 46 us; skew 500 ppm
I20260807 03:06:52.465193 30604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:52 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072012, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:52.468284 30604 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:06:52.469816 30604 webserver.cc:460] Webserver started at http://127.5.122.2:37909/ using document root <none> and password file <none>
I20260807 03:06:52.470556 30604 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:52.470682 30604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:52.476025 30604 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 03:06:52.478730 30622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:52.479976 30604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:06:52.480132 30604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "33a975be89644b7bb1cf3d9c41afac84"
format_stamp: "Formatted at 2026-08-07 03:06:06 on dist-test-slave-fgb3"
server_key: "4f571a779c5e1acf5fb5e30266593133"
server_key_iv: "0cda1ccff40c344d0be1dab68f55d57a"
server_key_version: "encryptionkey@0"
I20260807 03:06:52.481089 30604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:52.495296 30604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:52.499029 30604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:52.499280 30604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:52.500202 30604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:52.501500 30604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:52.501595 30604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:52.501694 30604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:52.501770 30604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:52.532522 30604 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:37601
I20260807 03:06:52.532590 30791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:37601 every 8 connection(s)
I20260807 03:06:52.533949 30604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:06:52.538599 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.536737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:38751)
0807 03:06:52.537450 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.537657 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":284,"threads_started":1}
W20260807 03:06:52.539278 30792 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:33843 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.62:33843: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
I20260807 03:06:52.539712 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 30604
I20260807 03:06:52.540206 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.539781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:56231)
0807 03:06:52.539987 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.540075 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 03:06:52.540477 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:34269
--local_ip_for_outbound_sockets=127.5.122.3
--tserver_master_addrs=127.5.122.62:33843
--webserver_port=42799
--webserver_interface=127.5.122.3
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:06:52.541114 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.540752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:40519)
0807 03:06:52.540884 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.540977 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 03:06:52.541337 30792 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 03:06:52.863864 30795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:06:52.864162 30795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:06:52.864288 30795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:06:52.871321 30795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:06:52.871441 30795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:06:52.871596 30795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:06:52.880968 30795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:34269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=42799
--tserver_master_addrs=127.5.122.62:33843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:06:52.882198 30795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:06:52.883502 30795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:06:52.896111 30804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:52.896183 30806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:06:52.896162 30809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:06:52.896979 30795 server_base.cc:1034] running on GCE node
I20260807 03:06:52.897545 30795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:06:52.898502 30795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:06:52.899696 30795 hybrid_clock.cc:648] HybridClock initialized: now 1786072012899651 us; error 37 us; skew 500 ppm
I20260807 03:06:52.899981 30795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:06:52 dist-test-slave-fgb3 krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072012, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:06:52.903031 30795 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:06:52.904438 30795 webserver.cc:460] Webserver started at http://127.5.122.3:42799/ using document root <none> and password file <none>
I20260807 03:06:52.905103 30795 fs_manager.cc:359] Metadata directory not provided
I20260807 03:06:52.905226 30795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:06:52.910405 30795 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:06:52.912974 30819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:52.914259 30795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:06:52.914433 30795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "40c33d52aab44873a803fc40e306d5a4"
format_stamp: "Formatted at 2026-08-07 03:06:07 on dist-test-slave-fgb3"
server_key: "698549b05a7d897a13d1806f7ea1c965"
server_key_iv: "78ff8ba02484bcbd6a12d1f64a528281"
server_key_version: "encryptionkey@0"
I20260807 03:06:52.915413 30795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:06:52.932947 30795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:06:52.936628 30795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:06:52.936873 30795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:06:52.937850 30795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:06:52.939203 30795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:06:52.939311 30795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:52.939420 30795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:06:52.939498 30795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:06:52.969247 30795 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:34269
I20260807 03:06:52.969316 31009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:34269 every 8 connection(s)
I20260807 03:06:52.970717 30795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:06:52.975376 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.973553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:56111)
0807 03:06:52.974268 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.974533 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":283,"threads_started":1}
W20260807 03:06:52.975975 31011 heartbeater.cc:644] Failed to heartbeat to 127.5.122.62:33843 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.62:33843: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
I20260807 03:06:52.976696 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.976401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:50139)
0807 03:06:52.976525 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.976591 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 03:06:52.977221 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 30795
I20260807 03:06:52.977511 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:52.977178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:59087)
0807 03:06:52.977310 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:52.977396 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 03:06:52.977731 31011 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 03:06:53.108480 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:53.107725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:44061)
0807 03:06:53.108190 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:53.108286 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":218,"threads_started":1}
I20260807 03:06:53.338332 31017 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:06:53.338719 31017 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:53.340472 31017 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601), 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
Aug 07 03:06:53 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
Aug 07 03:06:53 dist-test-slave-fgb3 krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:06:53.364768 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:53.340979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:53665 (local address 127.5.122.2:37601)
0807 03:06:53.341737 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:53.341752 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:53.341794 (+ 42us) server_negotiation.cc:407] Connection header received
0807 03:06:53.344006 (+ 2212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:53.344019 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:53.344507 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:53.344891 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:53.347138 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.349029 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.351600 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.352625 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.352820 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:53.357289 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:53.357303 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:53.357326 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:53.357377 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:53.360440 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:53.361803 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:53.361812 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:53.361821 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:53.362013 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:53.362429 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:53.362434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:53.362437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:53.362990 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:53.363175 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:53.364006 (+ 831us) server_negotiation.cc:299] Negotiation successful
0807 03:06:53.364342 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":457,"threads_started":1}
I20260807 03:06:53.364787 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:53.341614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:49877 (local address 127.5.122.3:34269)
0807 03:06:53.342430 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0807 03:06:53.342444 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:06:53.342484 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:06:53.344543 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:06:53.344554 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:06:53.344960 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:06:53.345279 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:06:53.347138 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.349029 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.351579 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.352625 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.352819 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:53.357488 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:06:53.357496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:06:53.357507 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:06:53.357546 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:06:53.360590 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:53.362163 (+ 1573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:53.362170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:53.362175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:53.362388 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:06:53.362768 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:06:53.362773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:06:53.362777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:06:53.363265 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:06:53.363427 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 03:06:53.364002 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 03:06:53.364376 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":424,"threads_started":1}
I20260807 03:06:53.364784 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:53.341185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:34269 (local address 127.5.122.1:49877)
0807 03:06:53.342031 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:53.342050 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:53.342164 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:53.345398 (+ 3234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:53.345403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:53.346017 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:53.346909 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:53.346938 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.349239 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.349245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:53.351422 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:53.351437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.352840 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.352846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:53.353174 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:53.354039 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:53.354075 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:53.357227 (+ 3152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:53.360730 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:53.360735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:53.360739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:53.361280 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:53.362500 (+ 1220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:53.362503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:53.362505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:53.362646 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:53.363399 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:53.363403 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:53.363541 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 03:06:53.363731 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 03:06:53.364320 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":171,"spinlock_wait_cycles":57472,"thread_start_us":569,"threads_started":1}
I20260807 03:06:53.366510 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:53.340772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:37601 (local address 127.5.122.1:53665)
0807 03:06:53.340949 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:53.340987 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 03:06:53.341550 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:06:53.344957 (+ 3407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:06:53.344964 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:06:53.345894 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:06:53.346909 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:53.346938 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.349244 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.349249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:53.351422 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:06:53.351437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:06:53.352855 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:06:53.352860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:06:53.353177 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:06:53.354033 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:06:53.354072 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:06:53.356905 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:06:53.360591 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:53.360601 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:53.360617 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:53.361269 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:53.362118 (+ 849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:06:53.362122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:06:53.362125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:06:53.362294 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:06:53.363134 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:06:53.363142 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:06:53.363430 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 03:06:53.363733 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 03:06:53.364470 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":513,"spinlock_wait_cycles":18048}
W20260807 03:06:53.368218 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
W20260807 03:06:53.369072 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
I20260807 03:06:53.369194 30433 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76c3712a0d23410f8c6bfda2b8b2db96; no voters: 33a975be89644b7bb1cf3d9c41afac84, 40c33d52aab44873a803fc40e306d5a4
I20260807 03:06:53.369545 31017 raft_consensus.cc:2726] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:06:53.542644 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:53.541789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:59195)
0807 03:06:53.542242 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:53.542406 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":211,"threads_started":1}
I20260807 03:06:53.978926 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:53.978201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:51391)
0807 03:06:53.978619 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:53.978719 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":216,"threads_started":1}
I20260807 03:06:54.109962 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:54.109293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:57259)
0807 03:06:54.109721 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:54.109800 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":240,"threads_started":1}
I20260807 03:06:54.544175 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:54.543504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:34069)
0807 03:06:54.543925 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:54.544018 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":220,"threads_started":1}
I20260807 03:06:54.980599 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:54.979804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:38161)
0807 03:06:54.980307 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:54.980416 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":271,"threads_started":1}
I20260807 03:06:55.111552 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:55.110752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:50321)
0807 03:06:55.111281 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:55.111358 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":220,"threads_started":1}
I20260807 03:06:55.470499 31032 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:06:55.470734 31032 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:55.471509 31032 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601), 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
W20260807 03:06:55.472584 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
W20260807 03:06:55.472785 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
I20260807 03:06:55.472879 30433 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76c3712a0d23410f8c6bfda2b8b2db96; no voters: 33a975be89644b7bb1cf3d9c41afac84, 40c33d52aab44873a803fc40e306d5a4
I20260807 03:06:55.473137 31032 raft_consensus.cc:2726] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:06:55.545625 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:55.544910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:57661)
0807 03:06:55.545371 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:55.545453 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":218,"threads_started":1}
I20260807 03:06:55.982065 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:55.981351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:40649)
0807 03:06:55.981813 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:55.981902 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":251,"threads_started":1}
I20260807 03:06:56.113060 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:56.112363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:40511)
0807 03:06:56.112809 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:56.112887 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":245,"threads_started":1}
I20260807 03:06:56.547175 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:56.546479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:57271)
0807 03:06:56.546915 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:56.546996 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":202,"threads_started":1}
I20260807 03:06:56.983749 31039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:56.983004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:33817)
0807 03:06:56.983464 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:56.983551 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":221,"threads_started":1}
I20260807 03:06:57.114831 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:57.114043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:33657)
0807 03:06:57.114557 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:57.114636 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":297,"threads_started":1}
I20260807 03:06:57.548802 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:57.548062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:48049)
0807 03:06:57.548540 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:57.548624 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":273,"threads_started":1}
I20260807 03:06:57.645838 31045 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:06:57.646035 31045 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:06:57.646807 31045 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601), 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
W20260807 03:06:57.647783 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
W20260807 03:06:57.647979 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
I20260807 03:06:57.648085 30433 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76c3712a0d23410f8c6bfda2b8b2db96; no voters: 33a975be89644b7bb1cf3d9c41afac84, 40c33d52aab44873a803fc40e306d5a4
I20260807 03:06:57.648346 31045 raft_consensus.cc:2726] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:06:57.985260 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:57.984597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:60911)
0807 03:06:57.985012 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:57.985097 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":215,"threads_started":1}
I20260807 03:06:58.116362 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:58.115589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:51983)
0807 03:06:58.116064 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:58.116169 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
I20260807 03:06:58.550472 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:58.549624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:45039)
0807 03:06:58.550134 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:58.550219 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":270,"threads_started":1}
I20260807 03:06:58.986847 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:58.986083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:50991)
0807 03:06:58.986560 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:58.986648 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":271,"threads_started":1}
I20260807 03:06:59.117861 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:59.117178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:58241)
0807 03:06:59.117590 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:59.117667 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":218,"threads_started":1}
I20260807 03:06:59.552122 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:59.551288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:49627)
0807 03:06:59.551838 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:59.551930 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":240,"threads_started":1}
I20260807 03:06:59.988505 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:06:59.987677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:54225)
0807 03:06:59.988197 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 03:06:59.988278 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":234,"threads_started":1}
I20260807 03:07:00.000986 31058 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:00.001159 31058 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:07:00.001724 31058 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601), 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
W20260807 03:07:00.002720 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
W20260807 03:07:00.002925 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
I20260807 03:07:00.003017 30433 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76c3712a0d23410f8c6bfda2b8b2db96; no voters: 33a975be89644b7bb1cf3d9c41afac84, 40c33d52aab44873a803fc40e306d5a4
I20260807 03:07:00.003268 31058 raft_consensus.cc:2726] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:07:00.119511 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:00.118684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:42861)
0807 03:07:00.119160 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:00.119281 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":244,"threads_started":1}
I20260807 03:07:00.553769 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:00.553063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:52721)
0807 03:07:00.553502 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:00.553584 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":240,"threads_started":1}
I20260807 03:07:00.990103 31061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:00.989349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:52199)
0807 03:07:00.989818 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:00.989909 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
I20260807 03:07:01.121022 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:01.120302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:58917)
0807 03:07:01.120755 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:01.120840 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":223,"threads_started":1}
I20260807 03:07:01.555475 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:01.554733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:55149)
0807 03:07:01.555223 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:01.555307 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":235,"threads_started":1}
I20260807 03:07:01.991695 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:01.990966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:42443)
0807 03:07:01.991436 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:01.991534 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":224,"threads_started":1}
I20260807 03:07:02.122706 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:02.122002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:49149)
0807 03:07:02.122443 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:02.122526 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":212,"threads_started":1}
I20260807 03:07:02.557226 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:02.556440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:51321)
0807 03:07:02.556937 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:02.557041 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":273,"threads_started":1}
I20260807 03:07:02.993332 31074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:02.992628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:58143)
0807 03:07:02.993067 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:02.993149 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":226,"threads_started":1}
I20260807 03:07:03.124233 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:03.123534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:59241)
0807 03:07:03.123971 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:03.124060 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":217,"threads_started":1}
I20260807 03:07:03.558848 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:03.558091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:45865)
0807 03:07:03.558598 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:03.558684 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":266,"threads_started":1}
I20260807 03:07:03.995008 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:03.994208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:40681)
0807 03:07:03.994760 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:03.994844 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":292,"threads_started":1}
I20260807 03:07:04.125772 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:04.125073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:49009)
0807 03:07:04.125523 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:04.125602 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":243,"threads_started":1}
I20260807 03:07:04.130620 31080 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:04.130806 31080 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:07:04.131470 31080 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601), 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
W20260807 03:07:04.132524 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
W20260807 03:07:04.132742 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
I20260807 03:07:04.132853 30433 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76c3712a0d23410f8c6bfda2b8b2db96; no voters: 33a975be89644b7bb1cf3d9c41afac84, 40c33d52aab44873a803fc40e306d5a4
I20260807 03:07:04.133096 31080 raft_consensus.cc:2726] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:07:04.560530 31082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:04.559744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:57209)
0807 03:07:04.560227 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:04.560342 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":252,"threads_started":1}
I20260807 03:07:04.996578 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:04.995870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:51043)
0807 03:07:04.996321 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:04.996403 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260807 03:07:05.127465 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:05.126730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:37207)
0807 03:07:05.127172 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:05.127258 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":233,"threads_started":1}
I20260807 03:07:05.562130 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:05.561423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:51499)
0807 03:07:05.561871 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:05.561965 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":218,"threads_started":1}
I20260807 03:07:05.998147 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:05.997432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:33445)
0807 03:07:05.997858 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:05.997967 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":209,"threads_started":1}
I20260807 03:07:06.128993 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:06.128247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:55737)
0807 03:07:06.128712 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:06.128802 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":244,"threads_started":1}
I20260807 03:07:06.332370 31091 raft_consensus.cc:482] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:06.332580 31091 raft_consensus.cc:504] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76c3712a0d23410f8c6bfda2b8b2db96" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 37953 } } peers { permanent_uuid: "33a975be89644b7bb1cf3d9c41afac84" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 37601 } } peers { permanent_uuid: "40c33d52aab44873a803fc40e306d5a4" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34269 } }
I20260807 03:07:06.333233 31091 leader_election.cc:288] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601), 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269)
W20260807 03:07:06.334298 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33a975be89644b7bb1cf3d9c41afac84 (127.5.122.2:37601): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
W20260807 03:07:06.334558 30433 leader_election.cc:341] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40c33d52aab44873a803fc40e306d5a4 (127.5.122.3:34269): Not found: Tablet not found: 2e3070c957504c43981987b64b5258ff
I20260807 03:07:06.334657 30433 leader_election.cc:302] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76c3712a0d23410f8c6bfda2b8b2db96; no voters: 33a975be89644b7bb1cf3d9c41afac84, 40c33d52aab44873a803fc40e306d5a4
I20260807 03:07:06.334910 31091 raft_consensus.cc:2726] T 2e3070c957504c43981987b64b5258ff P 76c3712a0d23410f8c6bfda2b8b2db96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 03:07:06.563738 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:06.562966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:38083)
0807 03:07:06.563449 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:06.563531 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":253,"threads_started":1}
I20260807 03:07:06.999694 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:06.998969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:37881)
0807 03:07:06.999408 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:06.999499 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":226,"threads_started":1}
I20260807 03:07:07.130874 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:07.130119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:49019)
0807 03:07:07.130591 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:07.130691 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260807 03:07:07.565694 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:07.564911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.2:37027)
0807 03:07:07.565368 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:07.565520 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":224,"threads_started":1}
I20260807 03:07:07.990016 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 30349
I20260807 03:07:08.001407 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:08.000706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.3:56935)
0807 03:07:08.001156 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:08.001245 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":234,"threads_started":1}
I20260807 03:07:08.132568 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:08.131776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33843 (local address 127.5.122.1:49007)
0807 03:07:08.132310 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:08.132402 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.62:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":247,"threads_started":1}
I20260807 03:07:08.404831 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:07.991137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.5.122.1:37953)
0807 03:07:07.991725 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:07.991736 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:08.387055 (+395319us) server_negotiation.cc:407] Connection header received
0807 03:07:08.387246 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:08.387258 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:08.387517 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:08.387672 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:08.389223 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:08.390261 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:08.391568 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:08.392134 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:08.392253 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:08.399111 (+ 6858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:08.399125 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:08.399141 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:08.399181 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:08.401889 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:08.402656 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:08.402664 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:08.402667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:08.402804 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:08.403183 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:08.403187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:08.403189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:08.403564 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:08.403893 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:08.404224 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 03:07:08.404499 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":239,"threads_started":1}
I20260807 03:07:08.406040 30589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:08.493129 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 30349
I20260807 03:07:08.517359 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 30604
I20260807 03:07:08.532430 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:08.518178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.5.122.2:37601)
0807 03:07:08.518668 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:08.518673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:08.518690 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:07:08.518746 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:08.518750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:08.518810 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:08.518897 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:08.520324 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:08.521064 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:08.522796 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:08.523406 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:08.523534 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:08.526771 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:08.526776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:08.526779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:08.526811 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:08.529652 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:08.530352 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:08.530357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:08.530362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:08.530506 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:08.530961 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:08.530967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:08.530971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:08.531330 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:08.531585 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:08.531953 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 03:07:08.532139 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
I20260807 03:07:08.533140 30783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:08.615025 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 30604
I20260807 03:07:08.637574 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 30795
I20260807 03:07:08.650907 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:08.638462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48970 (local address 127.5.122.3:34269)
0807 03:07:08.639076 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:08.639081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:08.639102 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:07:08.639165 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:08.639169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:08.639234 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:08.639347 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:08.640595 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:08.641484 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:08.642873 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:08.643324 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:08.643434 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:08.645850 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:08.645855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:08.645859 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:08.645894 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:08.648552 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:08.649191 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:08.649195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:08.649197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:08.649377 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:08.649689 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:08.649692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:08.649694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:08.649959 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:08.650194 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:08.650363 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 03:07:08.650551 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":375,"threads_started":1}
I20260807 03:07:08.651659 31005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:08.733982 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 30795
2026-08-07T03:07:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64752 ms)
[----------] 30 tests from SecurityITest (294956 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:08 dist-test-slave-fgb3 krb5kdc[31111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:08 dist-test-slave-fgb3 krb5kdc[31111](info): set up 2 sockets
Aug 07 03:07:08 dist-test-slave-fgb3 krb5kdc[31111](info): commencing operation
krb5kdc: starting...
W20260807 03:07:09.924189 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:10 dist-test-slave-fgb3 krb5kdc[31111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:07:10.199810 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:41783
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:41783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:10.517612 31127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:10.517884 31127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:10.517975 31127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:10.525027 31127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:10.525122 31127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:10.525167 31127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:10.525223 31127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:10.525265 31127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:10.534842 31127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:41783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:41783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:10.536073 31127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:10.537300 31127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:10.548370 31134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:10.548416 31135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:10.548578 31139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:10.549417 31127 server_base.cc:1034] running on GCE node
I20260807 03:07:10.550107 31127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:10.551486 31127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:10.552718 31127 hybrid_clock.cc:648] HybridClock initialized: now 1786072030552651 us; error 44 us; skew 500 ppm
I20260807 03:07:10.553052 31127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:10 dist-test-slave-fgb3 krb5kdc[31111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072030, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:10.556149 31127 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:07:10.557546 31127 webserver.cc:460] Webserver started at http://127.5.122.62:37225/ using document root <none> and password file <none>
I20260807 03:07:10.558215 31127 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:10.558358 31127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:10.558660 31127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:10.561522 31127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "0f5f531031464083bec541daac1a4799"
format_stamp: "Formatted at 2026-08-07 03:07:10 on dist-test-slave-fgb3"
server_key: "a08c9a8aae0239b77e2a6cede96ebd66"
server_key_iv: "dee0de06efadbe464aba5b0acb1400a0"
server_key_version: "encryptionkey@0"
I20260807 03:07:10.562263 31127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "0f5f531031464083bec541daac1a4799"
format_stamp: "Formatted at 2026-08-07 03:07:10 on dist-test-slave-fgb3"
server_key: "a08c9a8aae0239b77e2a6cede96ebd66"
server_key_iv: "dee0de06efadbe464aba5b0acb1400a0"
server_key_version: "encryptionkey@0"
I20260807 03:07:10.567037 31127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 03:07:10.570127 31151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:10.571298 31127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:07:10.571452 31127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "0f5f531031464083bec541daac1a4799"
format_stamp: "Formatted at 2026-08-07 03:07:10 on dist-test-slave-fgb3"
server_key: "a08c9a8aae0239b77e2a6cede96ebd66"
server_key_iv: "dee0de06efadbe464aba5b0acb1400a0"
server_key_version: "encryptionkey@0"
I20260807 03:07:10.571619 31127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:10.584200 31127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:10.588269 31127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:10.588532 31127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:10.605417 31127 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:41783
I20260807 03:07:10.605456 31229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:41783 every 8 connection(s)
I20260807 03:07:10.606736 31127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:10.610231 31230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:10.612654 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 31127
I20260807 03:07:10.612855 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:10.613492 5608 external_mini_cluster.cc:1428] Setting key 8aa6b0a08428139d540046c7c344974c
I20260807 03:07:10.622432 31230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799: Bootstrap starting.
I20260807 03:07:10.625591 31230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:10.626550 31230 log.cc:824] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799: Log is configured to *not* fsync() on all Append() calls
Aug 07 03:07:10 dist-test-slave-fgb3 krb5kdc[31111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:10.629105 31230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799: No bootstrap required, opened a new log
I20260807 03:07:10.637038 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:10.615444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56296 (local address 127.5.122.62:41783)
0807 03:07:10.616219 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:10.616233 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:10.616282 (+ 49us) server_negotiation.cc:407] Connection header received
0807 03:07:10.618484 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:10.618509 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:10.618970 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:10.619373 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:10.620622 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:10.622548 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:10.623935 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:10.624889 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:10.625068 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:10.629001 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:10.629012 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:10.629030 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:10.629069 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:10.632786 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:10.633409 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:10.633417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:10.633424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:10.633617 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:10.633966 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:10.633971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:10.633974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:10.634714 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:10.634987 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:10.635397 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 03:07:10.635806 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":326,"threads_started":1}
I20260807 03:07:10.638971 31230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5f531031464083bec541daac1a4799" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41783 } }
I20260807 03:07:10.639370 31230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:10.639479 31230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5f531031464083bec541daac1a4799, State: Initialized, Role: FOLLOWER
I20260807 03:07:10.639986 31230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5f531031464083bec541daac1a4799" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41783 } }
I20260807 03:07:10.640187 31230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:10.640280 31230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:10.640385 31230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:10.642306 31230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5f531031464083bec541daac1a4799" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41783 } }
I20260807 03:07:10.642781 31230 leader_election.cc:302] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f5f531031464083bec541daac1a4799; no voters:
I20260807 03:07:10.643325 31230 leader_election.cc:288] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:10.643424 31236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:10.644546 31236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [term 1 LEADER]: Becoming Leader. State: Replica: 0f5f531031464083bec541daac1a4799, State: Running, Role: LEADER
I20260807 03:07:10.645042 31236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5f531031464083bec541daac1a4799" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41783 } }
I20260807 03:07:10.645223 31230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:10.647711 31237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f5f531031464083bec541daac1a4799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5f531031464083bec541daac1a4799" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41783 } } }
I20260807 03:07:10.648202 31237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:10.648126 31238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f5f531031464083bec541daac1a4799. Latest consensus state: current_term: 1 leader_uuid: "0f5f531031464083bec541daac1a4799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5f531031464083bec541daac1a4799" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41783 } } }
I20260807 03:07:10.648429 31238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:10.649070 31245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:10.653635 31245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:10.661671 31245 catalog_manager.cc:1347] Generated new cluster ID: 3de316a181254b76a522393904aa50b4
I20260807 03:07:10.661798 31245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:10.695395 31245 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:10.696702 31245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:10.710357 31245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799: Generated new TSK 0
I20260807 03:07:10.711185 31245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:07:10.787678 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41783
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:11.141243 31261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:11.141522 31261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:11.141652 31261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:11.148983 31261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:11.149102 31261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:11.149264 31261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:07:11.159314 31261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:11.160533 31261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:11.161801 31261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:11.174185 31273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:11.174240 31278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:11.174211 31275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:11.174901 31261 server_base.cc:1034] running on GCE node
I20260807 03:07:11.175463 31261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:11.176404 31261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:11.177604 31261 hybrid_clock.cc:648] HybridClock initialized: now 1786072031177564 us; error 34 us; skew 500 ppm
I20260807 03:07:11.177891 31261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:11 dist-test-slave-fgb3 krb5kdc[31111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072031, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:11.180985 31261 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:07:11.182439 31261 webserver.cc:460] Webserver started at http://127.5.122.1:33705/ using document root <none> and password file <none>
I20260807 03:07:11.183090 31261 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:11.183207 31261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:11.183496 31261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:11.186306 31261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "6f440c465dc04f6f89df0283fa15f1bb"
format_stamp: "Formatted at 2026-08-07 03:07:11 on dist-test-slave-fgb3"
server_key: "0ebd79cfe8f1de844e54bbc00e939a74"
server_key_iv: "500b27afe2c8b4ea7f9945f7ba563bb0"
server_key_version: "encryptionkey@0"
I20260807 03:07:11.187122 31261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "6f440c465dc04f6f89df0283fa15f1bb"
format_stamp: "Formatted at 2026-08-07 03:07:11 on dist-test-slave-fgb3"
server_key: "0ebd79cfe8f1de844e54bbc00e939a74"
server_key_iv: "500b27afe2c8b4ea7f9945f7ba563bb0"
server_key_version: "encryptionkey@0"
I20260807 03:07:11.192091 31261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:07:11.195389 31288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:11.196610 31261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:07:11.196760 31261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "6f440c465dc04f6f89df0283fa15f1bb"
format_stamp: "Formatted at 2026-08-07 03:07:11 on dist-test-slave-fgb3"
server_key: "0ebd79cfe8f1de844e54bbc00e939a74"
server_key_iv: "500b27afe2c8b4ea7f9945f7ba563bb0"
server_key_version: "encryptionkey@0"
I20260807 03:07:11.196934 31261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:11.215719 31261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:11.219794 31261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:11.220068 31261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:11.221011 31261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:11.222366 31261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:11.222447 31261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:11.222570 31261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:11.222623 31261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:11.252812 31261 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:36791
I20260807 03:07:11.252894 31465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:36791 every 8 connection(s)
I20260807 03:07:11.254016 31261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:07:11.262712 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 31261
I20260807 03:07:11.262902 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:07:11.263548 5608 external_mini_cluster.cc:1428] Setting key 249753e5c2dbf4ae647e91ea24b9b05e
Aug 07 03:07:11 dist-test-slave-fgb3 krb5kdc[31111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072031, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:11.272406 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:11.256357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:46845 (local address 127.5.122.62:41783)
0807 03:07:11.256784 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:11.256788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:11.257544 (+ 756us) server_negotiation.cc:407] Connection header received
0807 03:07:11.259378 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:11.259384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:11.259472 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:11.259571 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:11.260974 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.261703 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.263342 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.263753 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.263877 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:11.267293 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:11.267298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:11.267301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:11.267342 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:11.269612 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:11.270267 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:11.270272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:11.270274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:11.270417 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:11.270823 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:11.270826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:11.270828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:11.271064 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:11.271193 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:11.271851 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 03:07:11.272066 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":220,"threads_started":1}
I20260807 03:07:11.273217 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:11.256704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:41783 (local address 127.5.122.1:46845)
0807 03:07:11.257386 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:11.257424 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:11.259120 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:11.259815 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:11.259823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:11.260288 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:11.260808 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:11.260825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.261858 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.261862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:11.263205 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:11.263217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.263980 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.263988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:11.264195 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:11.264795 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:11.264829 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:11.267093 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:11.269740 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:11.269748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:11.269762 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:11.270142 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:11.270518 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:11.270523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:11.270527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:11.270701 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:11.271186 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:11.271192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:11.271482 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 03:07:11.271707 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 03:07:11.272167 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":267,"threads_started":1}
I20260807 03:07:11.274639 31467 heartbeater.cc:347] Connected to a master server at 127.5.122.62:41783
I20260807 03:07:11.274910 31467 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:11.275593 31467 heartbeater.cc:511] Master 127.5.122.62:41783 requested a full tablet report, sending...
I20260807 03:07:11.277712 31177 ts_manager.cc:194] Registered new tserver with Master: 6f440c465dc04f6f89df0283fa15f1bb (127.5.122.1:36791)
I20260807 03:07:11.279124 31177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:46845
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:07:11.327353 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41783
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:11.664574 31482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:11.664861 31482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:11.665011 31482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:11.672197 31482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:11.672329 31482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:11.672490 31482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:07:11.682427 31482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:11.683723 31482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:11.685003 31482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:11.697340 31489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:11.697381 31490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:11.697384 31492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:11.698238 31482 server_base.cc:1034] running on GCE node
I20260807 03:07:11.698851 31482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:11.699716 31482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:11.700922 31482 hybrid_clock.cc:648] HybridClock initialized: now 1786072031700859 us; error 52 us; skew 500 ppm
I20260807 03:07:11.701292 31482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:11 dist-test-slave-fgb3 krb5kdc[31111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072031, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:11.704262 31482 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:07:11.705659 31482 webserver.cc:460] Webserver started at http://127.5.122.2:44583/ using document root <none> and password file <none>
I20260807 03:07:11.706301 31482 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:11.706463 31482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:11.706764 31482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:11.709561 31482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1"
format_stamp: "Formatted at 2026-08-07 03:07:11 on dist-test-slave-fgb3"
server_key: "489a5b7b2026591ccc4ef6d8e7ddd09f"
server_key_iv: "bdd96674e017bbb2ea348ae259a66a1e"
server_key_version: "encryptionkey@0"
I20260807 03:07:11.710337 31482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1"
format_stamp: "Formatted at 2026-08-07 03:07:11 on dist-test-slave-fgb3"
server_key: "489a5b7b2026591ccc4ef6d8e7ddd09f"
server_key_iv: "bdd96674e017bbb2ea348ae259a66a1e"
server_key_version: "encryptionkey@0"
I20260807 03:07:11.715230 31482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 03:07:11.718552 31501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:11.719731 31482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:07:11.719894 31482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1"
format_stamp: "Formatted at 2026-08-07 03:07:11 on dist-test-slave-fgb3"
server_key: "489a5b7b2026591ccc4ef6d8e7ddd09f"
server_key_iv: "bdd96674e017bbb2ea348ae259a66a1e"
server_key_version: "encryptionkey@0"
I20260807 03:07:11.720070 31482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:11.750195 31482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:11.753961 31482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:11.754233 31482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:11.755224 31482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:11.756543 31482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:11.756623 31482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:11.756736 31482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:11.756805 31482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:11.787384 31482 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:39699
I20260807 03:07:11.787437 31662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:39699 every 8 connection(s)
I20260807 03:07:11.788599 31482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:07:11.796563 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 31482
I20260807 03:07:11.796726 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:07:11.797155 5608 external_mini_cluster.cc:1428] Setting key 62b071510a0c7336e664dcf2cdf7fab5
Aug 07 03:07:11 dist-test-slave-fgb3 krb5kdc[31111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072031, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:11.807132 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:11.790919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:35855 (local address 127.5.122.62:41783)
0807 03:07:11.791324 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:11.791329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:11.792123 (+ 794us) server_negotiation.cc:407] Connection header received
0807 03:07:11.793877 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:11.793883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:11.793959 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:11.794053 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:11.795483 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.796195 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.797753 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.798339 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.798461 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:11.802031 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:11.802035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:11.802038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:11.802069 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:11.804359 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:11.805002 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:11.805006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:11.805009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:11.805129 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:11.805539 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:11.805543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:11.805545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:11.805795 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:11.805924 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:11.806554 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 03:07:11.806765 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":202,"threads_started":1}
I20260807 03:07:11.807930 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:11.791322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:41783 (local address 127.5.122.2:35855)
0807 03:07:11.791977 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:11.792013 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:11.793624 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:11.794282 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:11.794289 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:11.794791 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:11.795311 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:11.795327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.796358 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.796362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:11.797618 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:11.797630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:11.798558 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:11.798566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:11.798784 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:11.799487 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:11.799508 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:11.801822 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:11.804493 (+ 2671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:11.804502 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:11.804515 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:11.804887 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:11.805234 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:11.805238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:11.805242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:11.805422 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:11.805918 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:11.805925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:11.806207 (+ 282us) client_negotiation.cc:769] Sending connection context
0807 03:07:11.806466 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 03:07:11.806921 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":270,"threads_started":1}
I20260807 03:07:11.809154 31664 heartbeater.cc:347] Connected to a master server at 127.5.122.62:41783
I20260807 03:07:11.809407 31664 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:11.810062 31664 heartbeater.cc:511] Master 127.5.122.62:41783 requested a full tablet report, sending...
I20260807 03:07:11.811405 31177 ts_manager.cc:194] Registered new tserver with Master: d954b2e41a7f42d2a4de1b8f47bc0fc1 (127.5.122.2:39699)
I20260807 03:07:11.812179 31177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:35855
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:07:11.859431 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41783
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:12.202526 31675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:12.202790 31675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:12.202913 31675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:12.210054 31675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:12.210176 31675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:12.210354 31675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:07:12.220265 31675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:12.221573 31675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:12.222927 31675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:12.235584 31682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:12.235671 31684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:12.235625 31686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:12.236356 31675 server_base.cc:1034] running on GCE node
I20260807 03:07:12.236935 31675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:12.237828 31675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:12.239022 31675 hybrid_clock.cc:648] HybridClock initialized: now 1786072032238957 us; error 58 us; skew 500 ppm
I20260807 03:07:12.239316 31675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:12 dist-test-slave-fgb3 krb5kdc[31111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072032, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:12.242249 31675 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:07:12.243590 31675 webserver.cc:460] Webserver started at http://127.5.122.3:38737/ using document root <none> and password file <none>
I20260807 03:07:12.244225 31675 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:12.244340 31675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:12.244628 31675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:12.247525 31675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "ebb94493b5844921bf23895eaecf1f35"
format_stamp: "Formatted at 2026-08-07 03:07:12 on dist-test-slave-fgb3"
server_key: "7c20f8c917c27a19b336f5c326e3a9ef"
server_key_iv: "6c5184bb0eba47c9735b3bc6dcbaecd6"
server_key_version: "encryptionkey@0"
I20260807 03:07:12.248265 31675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "ebb94493b5844921bf23895eaecf1f35"
format_stamp: "Formatted at 2026-08-07 03:07:12 on dist-test-slave-fgb3"
server_key: "7c20f8c917c27a19b336f5c326e3a9ef"
server_key_iv: "6c5184bb0eba47c9735b3bc6dcbaecd6"
server_key_version: "encryptionkey@0"
I20260807 03:07:12.253453 31675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 03:07:12.257714 31697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:12.258980 31675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:07:12.259120 31675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "ebb94493b5844921bf23895eaecf1f35"
format_stamp: "Formatted at 2026-08-07 03:07:12 on dist-test-slave-fgb3"
server_key: "7c20f8c917c27a19b336f5c326e3a9ef"
server_key_iv: "6c5184bb0eba47c9735b3bc6dcbaecd6"
server_key_version: "encryptionkey@0"
I20260807 03:07:12.259302 31675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:12.272416 31675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:12.276134 31675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:12.276386 31675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:12.277307 31675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:12.278599 31675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:12.278672 31675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:12.278784 31675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:12.278846 31675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:12.281694 31467 heartbeater.cc:503] Master 127.5.122.62:41783 was elected leader, sending a full tablet report...
I20260807 03:07:12.309546 31675 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:34949
I20260807 03:07:12.309590 31857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:34949 every 8 connection(s)
I20260807 03:07:12.310835 31675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:07:12.317830 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 31675
I20260807 03:07:12.318063 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:07:12.318612 5608 external_mini_cluster.cc:1428] Setting key 560ad2e33de85033991cdfe90cc983c5
Aug 07 03:07:12 dist-test-slave-fgb3 krb5kdc[31111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072032, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:12.329836 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.313336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:54741 (local address 127.5.122.62:41783)
0807 03:07:12.313774 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.313778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.314685 (+ 907us) server_negotiation.cc:407] Connection header received
0807 03:07:12.316459 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.316464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.316539 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.316633 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:12.318364 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.319189 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.321099 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.321557 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.321650 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.324894 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:12.324899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:12.324902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:12.324933 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:12.327445 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.328070 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.328074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.328077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.328188 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.328519 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.328522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.328524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.328774 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:12.328902 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.329409 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.329559 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":214,"threads_started":1}
I20260807 03:07:12.330725 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.313815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:41783 (local address 127.5.122.3:54741)
0807 03:07:12.314533 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:12.314573 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:12.316194 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:12.316867 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:12.316874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:12.317358 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:12.318144 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.318172 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.319352 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.319356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.320970 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.320986 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.321757 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.321763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.321949 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.322542 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:12.322563 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:12.324705 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:12.327571 (+ 2866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:12.327578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:12.327588 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:12.327983 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:12.328263 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:12.328266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:12.328268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:12.328444 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:12.328860 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:12.328865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:12.329137 (+ 272us) client_negotiation.cc:769] Sending connection context
0807 03:07:12.329345 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 03:07:12.329689 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":266,"threads_started":1}
I20260807 03:07:12.332010 31858 heartbeater.cc:347] Connected to a master server at 127.5.122.62:41783
I20260807 03:07:12.332307 31858 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:12.333104 31858 heartbeater.cc:511] Master 127.5.122.62:41783 requested a full tablet report, sending...
I20260807 03:07:12.334483 31177 ts_manager.cc:194] Registered new tserver with Master: ebb94493b5844921bf23895eaecf1f35 (127.5.122.3:34949)
I20260807 03:07:12.335254 31177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:54741
I20260807 03:07:12.345749 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:07:12.362434 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.350348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56308 (local address 127.5.122.62:41783)
0807 03:07:12.350540 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.350546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.350990 (+ 444us) server_negotiation.cc:407] Connection header received
0807 03:07:12.351119 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.351122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.351188 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.351267 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:12.352648 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.353377 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.354771 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.355194 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.355305 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.357587 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:12.357592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:12.357595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:12.357626 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:12.360250 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.360806 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.360809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.360812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.360924 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.361204 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.361206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.361208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.361522 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:12.361698 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.361963 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.362133 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 03:07:12.366304 31177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:07:12.373016 31177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:07:12.407557 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.385516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.5.122.3:34949)
0807 03:07:12.397954 (+ 12438us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.397963 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.400164 (+ 2201us) server_negotiation.cc:407] Connection header received
0807 03:07:12.400301 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.400308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.400491 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.400658 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:12.401421 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.403116 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.405521 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.406458 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.406568 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.406799 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.407065 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.407220 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12259,"thread_start_us":12083,"threads_started":1}
I20260807 03:07:12.407858 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.384852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39699 (local address 127.0.0.1:45166)
0807 03:07:12.399894 (+ 15042us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:12.399916 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:12.400054 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:12.400918 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:12.400923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:12.400962 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:12.401410 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.401424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.403303 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.403307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.404912 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.404925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.406985 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.406992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.407169 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.407224 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 03:07:12.407316 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 03:07:12.407446 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14849,"mutex_wait_us":52,"spinlock_wait_cycles":768,"thread_start_us":13319,"threads_started":1}
I20260807 03:07:12.407881 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.384900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.5.122.1:36791)
0807 03:07:12.397584 (+ 12684us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.397594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.399724 (+ 2130us) server_negotiation.cc:407] Connection header received
0807 03:07:12.399893 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.399902 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.400175 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.400368 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:12.401024 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.402491 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.405187 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.406562 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.406666 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.406871 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.407420 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.407576 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12479,"thread_start_us":12306,"threads_started":1}
I20260807 03:07:12.408390 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.385161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:34949 (local address 127.0.0.1:42784)
0807 03:07:12.400033 (+ 14872us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:12.400049 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:12.400167 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:12.400734 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:12.400738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:12.400769 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:12.401231 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.401240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.403248 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.403253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.405383 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.405392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.406638 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.406644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.406826 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.406871 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 03:07:12.407011 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 03:07:12.407200 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14758,"mutex_wait_us":40,"thread_start_us":14535,"threads_started":1}
I20260807 03:07:12.408691 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.385233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.5.122.2:39699)
0807 03:07:12.399474 (+ 14241us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.399483 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.400029 (+ 546us) server_negotiation.cc:407] Connection header received
0807 03:07:12.400240 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.400247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.400462 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.400646 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:12.401572 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.403115 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.405059 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.406504 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.406635 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.406817 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.408253 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.408424 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14070,"thread_start_us":13944,"threads_started":1}
I20260807 03:07:12.409198 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.384829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:36791 (local address 127.0.0.1:37832)
0807 03:07:12.398617 (+ 13788us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:12.398664 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:12.398872 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:12.400377 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:12.400384 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:12.400418 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:12.400829 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.400841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.402644 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.402648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.405022 (+ 2374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.405037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.407002 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.407008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.407177 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.407233 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 03:07:12.407308 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 03:07:12.408842 (+ 1534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13516,"mutex_wait_us":1448,"thread_start_us":14683,"threads_started":1}
I20260807 03:07:12.411906 31768 tablet_service.cc:1467] Processing CreateTablet for tablet c74b32135e1d4964bcca73bc1808cb89 (DEFAULT_TABLE table=test-table [id=5f5310f0481b4eae971524c5f773a768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:12.412155 31570 tablet_service.cc:1467] Processing CreateTablet for tablet c74b32135e1d4964bcca73bc1808cb89 (DEFAULT_TABLE table=test-table [id=5f5310f0481b4eae971524c5f773a768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:12.412389 31363 tablet_service.cc:1467] Processing CreateTablet for tablet c74b32135e1d4964bcca73bc1808cb89 (DEFAULT_TABLE table=test-table [id=5f5310f0481b4eae971524c5f773a768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:12.413143 31768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c74b32135e1d4964bcca73bc1808cb89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:12.413501 31363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c74b32135e1d4964bcca73bc1808cb89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:12.413512 31570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c74b32135e1d4964bcca73bc1808cb89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:12.425199 31889 tablet_bootstrap.cc:492] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Bootstrap starting.
I20260807 03:07:12.427414 31892 tablet_bootstrap.cc:492] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Bootstrap starting.
I20260807 03:07:12.428742 31893 tablet_bootstrap.cc:492] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Bootstrap starting.
I20260807 03:07:12.428927 31889 tablet_bootstrap.cc:654] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:12.430303 31889 log.cc:824] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:12.430702 31892 tablet_bootstrap.cc:654] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:12.431778 31893 tablet_bootstrap.cc:654] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:12.431931 31892 log.cc:824] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:12.432948 31893 log.cc:824] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:12.434309 31889 tablet_bootstrap.cc:492] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: No bootstrap required, opened a new log
I20260807 03:07:12.434665 31889 ts_tablet_manager.cc:1397] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 03:07:12.434713 31892 tablet_bootstrap.cc:492] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: No bootstrap required, opened a new log
I20260807 03:07:12.434948 31892 ts_tablet_manager.cc:1397] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 03:07:12.436347 31893 tablet_bootstrap.cc:492] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: No bootstrap required, opened a new log
I20260807 03:07:12.436600 31893 ts_tablet_manager.cc:1397] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 03:07:12.442233 31892 raft_consensus.cc:348] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.442638 31892 raft_consensus.cc:374] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:12.442734 31892 raft_consensus.cc:729] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d954b2e41a7f42d2a4de1b8f47bc0fc1, State: Initialized, Role: FOLLOWER
I20260807 03:07:12.443276 31892 consensus_queue.cc:260] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.444279 31664 heartbeater.cc:503] Master 127.5.122.62:41783 was elected leader, sending a full tablet report...
I20260807 03:07:12.444929 31892 ts_tablet_manager.cc:1428] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260807 03:07:12.444782 31889 raft_consensus.cc:348] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.445235 31889 raft_consensus.cc:374] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:12.445317 31889 raft_consensus.cc:729] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebb94493b5844921bf23895eaecf1f35, State: Initialized, Role: FOLLOWER
I20260807 03:07:12.446031 31889 consensus_queue.cc:260] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.446532 31893 raft_consensus.cc:348] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.446950 31893 raft_consensus.cc:374] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:12.447101 31893 raft_consensus.cc:729] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f440c465dc04f6f89df0283fa15f1bb, State: Initialized, Role: FOLLOWER
I20260807 03:07:12.447712 31893 consensus_queue.cc:260] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.450949 31858 heartbeater.cc:503] Master 127.5.122.62:41783 was elected leader, sending a full tablet report...
I20260807 03:07:12.451591 31889 ts_tablet_manager.cc:1428] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
I20260807 03:07:12.451834 31893 ts_tablet_manager.cc:1428] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 03:07:12.466488 31901 raft_consensus.cc:482] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:12.466717 31901 raft_consensus.cc:504] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.468016 31901 leader_election.cc:288] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f440c465dc04f6f89df0283fa15f1bb (127.5.122.1:36791), ebb94493b5844921bf23895eaecf1f35 (127.5.122.3:34949)
I20260807 03:07:12.474340 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.468385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:40303 (local address 127.5.122.3:34949)
0807 03:07:12.468569 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.468575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.469303 (+ 728us) server_negotiation.cc:407] Connection header received
0807 03:07:12.469435 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.469439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.469519 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.469610 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:12.470190 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.470934 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.472659 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.473391 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.473491 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.473584 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.473968 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.474088 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:07:12.474349 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.468375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:43757 (local address 127.5.122.1:36791)
0807 03:07:12.468567 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.468574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.469189 (+ 615us) server_negotiation.cc:407] Connection header received
0807 03:07:12.469334 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.469340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.469407 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.469477 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:12.470071 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.470846 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.472645 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.473441 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.473574 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.473665 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.473959 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.474082 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:07:12.474550 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.468154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:36791 (local address 127.5.122.2:43757)
0807 03:07:12.469091 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:12.469106 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:12.469215 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:12.469507 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:12.469512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:12.469533 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:12.469927 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.469939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.470973 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.470978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.472502 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.472514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.473666 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.473671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.473800 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.473844 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 03:07:12.473918 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 03:07:12.474075 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":46,"thread_start_us":612,"threads_started":1}
I20260807 03:07:12.474865 31911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.468385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:34949 (local address 127.5.122.2:40303)
0807 03:07:12.469200 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:12.469216 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:12.469334 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:12.469635 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:12.469638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:12.469668 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:12.470029 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.470040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.471073 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.471077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.472516 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:12.472529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.473615 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.473621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:12.473789 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.473841 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 03:07:12.473920 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 03:07:12.474073 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":41,"thread_start_us":421,"threads_started":1}
I20260807 03:07:12.475517 31402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74b32135e1d4964bcca73bc1808cb89" candidate_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f440c465dc04f6f89df0283fa15f1bb" is_pre_election: true
I20260807 03:07:12.475642 31803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74b32135e1d4964bcca73bc1808cb89" candidate_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebb94493b5844921bf23895eaecf1f35" is_pre_election: true
I20260807 03:07:12.475991 31402 raft_consensus.cc:2454] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d954b2e41a7f42d2a4de1b8f47bc0fc1 in term 0.
I20260807 03:07:12.476078 31803 raft_consensus.cc:2454] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d954b2e41a7f42d2a4de1b8f47bc0fc1 in term 0.
I20260807 03:07:12.476593 31502 leader_election.cc:302] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f440c465dc04f6f89df0283fa15f1bb, d954b2e41a7f42d2a4de1b8f47bc0fc1; no voters:
I20260807 03:07:12.476922 31901 raft_consensus.cc:2781] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:07:12.477039 31901 raft_consensus.cc:482] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:07:12.477145 31901 raft_consensus.cc:3037] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:12.479257 31901 raft_consensus.cc:504] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.479781 31901 leader_election.cc:288] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [CANDIDATE]: Term 1 election: Requested vote from peers 6f440c465dc04f6f89df0283fa15f1bb (127.5.122.1:36791), ebb94493b5844921bf23895eaecf1f35 (127.5.122.3:34949)
I20260807 03:07:12.480113 31402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74b32135e1d4964bcca73bc1808cb89" candidate_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f440c465dc04f6f89df0283fa15f1bb"
I20260807 03:07:12.480196 31803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c74b32135e1d4964bcca73bc1808cb89" candidate_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebb94493b5844921bf23895eaecf1f35"
I20260807 03:07:12.480273 31402 raft_consensus.cc:3037] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:12.480355 31803 raft_consensus.cc:3037] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:12.482360 31402 raft_consensus.cc:2454] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d954b2e41a7f42d2a4de1b8f47bc0fc1 in term 1.
I20260807 03:07:12.482430 31803 raft_consensus.cc:2454] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d954b2e41a7f42d2a4de1b8f47bc0fc1 in term 1.
I20260807 03:07:12.482703 31502 leader_election.cc:302] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f440c465dc04f6f89df0283fa15f1bb, d954b2e41a7f42d2a4de1b8f47bc0fc1; no voters:
I20260807 03:07:12.482986 31901 raft_consensus.cc:2781] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:12.483428 31901 raft_consensus.cc:686] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 1 LEADER]: Becoming Leader. State: Replica: d954b2e41a7f42d2a4de1b8f47bc0fc1, State: Running, Role: LEADER
I20260807 03:07:12.483901 31901 consensus_queue.cc:237] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } }
I20260807 03:07:12.488018 31177 catalog_manager.cc:5495] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d954b2e41a7f42d2a4de1b8f47bc0fc1 (127.5.122.2). New cstate: current_term: 1 leader_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d954b2e41a7f42d2a4de1b8f47bc0fc1" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 } health_report { overall_health: UNKNOWN } } }
W20260807 03:07:12.506219 31469 tablet.cc:2369] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:07:12.518191 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.513562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.5.122.2:39699)
0807 03:07:12.513722 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.513726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.513864 (+ 138us) server_negotiation.cc:407] Connection header received
0807 03:07:12.514029 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.514033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.514102 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.514214 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:07:12.514555 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.515224 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.516477 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.516859 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.516941 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.517342 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:07:12.517778 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:07:12.517878 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.517917 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.517975 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 03:07:12.525465 31803 raft_consensus.cc:1261] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 1 FOLLOWER]: Refusing update from remote peer d954b2e41a7f42d2a4de1b8f47bc0fc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:12.525475 31402 raft_consensus.cc:1261] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 1 FOLLOWER]: Refusing update from remote peer d954b2e41a7f42d2a4de1b8f47bc0fc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:12.526378 31913 consensus_queue.cc:1035] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f440c465dc04f6f89df0283fa15f1bb" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:12.526683 31901 consensus_queue.cc:1035] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebb94493b5844921bf23895eaecf1f35" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 34949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:12.550949 31175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:12.551280 31175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:12.554975 31175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f5f531031464083bec541daac1a4799: Sending DeleteTablet for 3 replicas of tablet c74b32135e1d4964bcca73bc1808cb89
I20260807 03:07:12.555994 31363 tablet_service.cc:1514] Processing DeleteTablet for tablet c74b32135e1d4964bcca73bc1808cb89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:12 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:37832
I20260807 03:07:12.556046 31570 tablet_service.cc:1514] Processing DeleteTablet for tablet c74b32135e1d4964bcca73bc1808cb89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:12 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:45166
I20260807 03:07:12.556542 31768 tablet_service.cc:1514] Processing DeleteTablet for tablet c74b32135e1d4964bcca73bc1808cb89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:12 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:42784
I20260807 03:07:12.558545 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 31261
I20260807 03:07:12.566797 31940 tablet_replica.cc:330] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: stopping tablet replica
I20260807 03:07:12.567018 31938 tablet_replica.cc:330] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: stopping tablet replica
I20260807 03:07:12.567400 31938 raft_consensus.cc:2229] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:12.567401 31940 raft_consensus.cc:2229] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:12.567916 31940 raft_consensus.cc:2258] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:12.567915 31938 raft_consensus.cc:2258] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:12.569952 31938 ts_tablet_manager.cc:1905] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:12.569941 31940 ts_tablet_manager.cc:1905] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:12.576797 31940 ts_tablet_manager.cc:1918] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:12.576797 31938 ts_tablet_manager.cc:1918] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:12.576952 31938 log.cc:1196] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/c74b32135e1d4964bcca73bc1808cb89
I20260807 03:07:12.577062 31940 log.cc:1196] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/c74b32135e1d4964bcca73bc1808cb89
I20260807 03:07:12.577535 31938 ts_tablet_manager.cc:1939] T c74b32135e1d4964bcca73bc1808cb89 P 6f440c465dc04f6f89df0283fa15f1bb: Deleting consensus metadata
I20260807 03:07:12.577637 31940 ts_tablet_manager.cc:1939] T c74b32135e1d4964bcca73bc1808cb89 P ebb94493b5844921bf23895eaecf1f35: Deleting consensus metadata
I20260807 03:07:12.579372 31152 catalog_manager.cc:4841] TS 6f440c465dc04f6f89df0283fa15f1bb (127.5.122.1:36791): tablet c74b32135e1d4964bcca73bc1808cb89 (table test-table [id=5f5310f0481b4eae971524c5f773a768]) successfully deleted
I20260807 03:07:12.579804 31156 catalog_manager.cc:4841] TS ebb94493b5844921bf23895eaecf1f35 (127.5.122.3:34949): tablet c74b32135e1d4964bcca73bc1808cb89 (table test-table [id=5f5310f0481b4eae971524c5f773a768]) successfully deleted
I20260807 03:07:12.580881 31939 tablet_replica.cc:330] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: stopping tablet replica
I20260807 03:07:12.581346 31939 raft_consensus.cc:2229] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:07:12.582026 31939 raft_consensus.cc:2258] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:12.583848 31939 ts_tablet_manager.cc:1905] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 03:07:12 dist-test-slave-fgb3 krb5kdc[31111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:07:12.590596 31939 ts_tablet_manager.cc:1918] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:12.590782 31939 log.cc:1196] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/c74b32135e1d4964bcca73bc1808cb89
I20260807 03:07:12.591264 31939 ts_tablet_manager.cc:1939] T c74b32135e1d4964bcca73bc1808cb89 P d954b2e41a7f42d2a4de1b8f47bc0fc1: Deleting consensus metadata
I20260807 03:07:12.593032 31155 catalog_manager.cc:4841] TS d954b2e41a7f42d2a4de1b8f47bc0fc1 (127.5.122.2:39699): tablet c74b32135e1d4964bcca73bc1808cb89 (table test-table [id=5f5310f0481b4eae971524c5f773a768]) successfully deleted
I20260807 03:07:12.597793 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.559818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.5.122.1:36791)
0807 03:07:12.560000 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.560006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.580235 (+ 20229us) server_negotiation.cc:407] Connection header received
0807 03:07:12.580412 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.580417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.580507 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.580599 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:12.582908 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.583942 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.585791 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.586453 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.586586 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.591801 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:12.591817 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:12.591835 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:12.591885 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:12.594958 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.595915 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.595922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.595924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.596055 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.596433 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.596436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.596438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.596781 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:12.597002 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.597342 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.597513 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:07:12.598505 31459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:12.687182 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 31261
I20260807 03:07:12.716648 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 31482
Aug 07 03:07:12 dist-test-slave-fgb3 krb5kdc[31111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:07:12.732121 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.717523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45174 (local address 127.5.122.2:39699)
0807 03:07:12.717748 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.717754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.717772 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:07:12.717829 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.717832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.717891 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.718011 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:12.719310 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.720108 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.721336 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.721775 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.721884 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.726515 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:12.726529 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:12.726544 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:12.726580 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:12.729448 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.730201 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.730207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.730209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.730382 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.730748 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.730751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.730753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.731082 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:12.731345 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.731625 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.731777 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 03:07:12.732900 31658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:12.841100 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 31482
I20260807 03:07:12.866014 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 31675
Aug 07 03:07:12 dist-test-slave-fgb3 krb5kdc[31111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:07:12.880095 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:12.866756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.5.122.3:34949)
0807 03:07:12.866936 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:12.866940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:12.866960 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:07:12.867077 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:12.867081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:12.867146 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:12.867224 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:12.868462 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.869155 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.870516 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:12.870886 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:12.870973 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:12.875108 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:12.875121 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:12.875137 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:12.875169 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:12.877576 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.878186 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.878193 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.878195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.878564 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:12.878880 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:12.878883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:12.878885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:12.879189 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:12.879386 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:12.879631 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 03:07:12.879791 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:07:12.880718 31852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:12.966439 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 31675
I20260807 03:07:12.987448 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 31127
I20260807 03:07:12.988580 31224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:13.096642 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 31127
2026-08-07T03:07:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4363 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T03:07:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:13Z Disabled control of system clock
I20260807 03:07:13.181331 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:41885
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:41885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 03:07:13.502686 31952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:13.502986 31952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:13.503093 31952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:13.510195 31952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:13.510300 31952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:13.510370 31952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:13.510452 31952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:13.510506 31952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:13.519594 31952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:41885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:41885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:13.520829 31952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:13.522646 31952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:13.539927 31957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:13.542160 31958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:13.544361 31960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:13.545969 31952 server_base.cc:1034] running on GCE node
I20260807 03:07:13.546880 31952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:13.548357 31952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:13.549582 31952 hybrid_clock.cc:648] HybridClock initialized: now 1786072033549527 us; error 47 us; skew 500 ppm
I20260807 03:07:13.549944 31952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:13.555369 31952 webserver.cc:460] Webserver started at http://127.5.122.62:43901/ using document root <none> and password file <none>
I20260807 03:07:13.556069 31952 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:13.556173 31952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:13.556499 31952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:13.559464 31952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "badb62e50c344bf09319d802c15a8a8e"
format_stamp: "Formatted at 2026-08-07 03:07:13 on dist-test-slave-fgb3"
server_key: "e97504fe40c9789011e1eedefbb68ed1"
server_key_iv: "9374a1b35c3e341b2e5a3455ecb1aee0"
server_key_version: "encryptionkey@0"
I20260807 03:07:13.560254 31952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "badb62e50c344bf09319d802c15a8a8e"
format_stamp: "Formatted at 2026-08-07 03:07:13 on dist-test-slave-fgb3"
server_key: "e97504fe40c9789011e1eedefbb68ed1"
server_key_iv: "9374a1b35c3e341b2e5a3455ecb1aee0"
server_key_version: "encryptionkey@0"
I20260807 03:07:13.565318 31952 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 03:07:13.568879 31971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:13.570155 31952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 03:07:13.570367 31952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "badb62e50c344bf09319d802c15a8a8e"
format_stamp: "Formatted at 2026-08-07 03:07:13 on dist-test-slave-fgb3"
server_key: "e97504fe40c9789011e1eedefbb68ed1"
server_key_iv: "9374a1b35c3e341b2e5a3455ecb1aee0"
server_key_version: "encryptionkey@0"
I20260807 03:07:13.570566 31952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:13.594255 31952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:13.595115 31952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:13.595382 31952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:13.612500 31952 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:41885
I20260807 03:07:13.612524 32045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:41885 every 8 connection(s)
I20260807 03:07:13.613749 31952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:13.615640 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 31952
I20260807 03:07:13.615823 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:13.616448 5608 external_mini_cluster.cc:1428] Setting key c35f2ed46ae352ba3bcbc4f4d19ca4fb
I20260807 03:07:13.618091 32046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:13.630003 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:13.618373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.5.122.62:41885)
0807 03:07:13.619314 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:13.619324 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:13.619364 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:07:13.620896 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:13.620915 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:13.621351 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:13.621670 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:13.622259 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:13.623629 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:13.625555 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:13.626215 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:13.626424 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:13.627651 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:13.627662 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:13.627683 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:13.628337 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:13.628415 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:13.628439 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:13.628536 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:13.628784 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 03:07:13.629102 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":351,"threads_started":1}
I20260807 03:07:13.630752 32046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e: Bootstrap starting.
I20260807 03:07:13.633865 32046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:13.634836 32046 log.cc:824] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:13.637516 32046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e: No bootstrap required, opened a new log
I20260807 03:07:13.644807 32046 raft_consensus.cc:348] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badb62e50c344bf09319d802c15a8a8e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41885 } }
I20260807 03:07:13.645133 32046 raft_consensus.cc:374] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:13.645236 32046 raft_consensus.cc:729] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: badb62e50c344bf09319d802c15a8a8e, State: Initialized, Role: FOLLOWER
I20260807 03:07:13.645746 32046 consensus_queue.cc:260] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badb62e50c344bf09319d802c15a8a8e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41885 } }
I20260807 03:07:13.645964 32046 raft_consensus.cc:388] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:13.646099 32046 raft_consensus.cc:482] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:13.646245 32046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:13.648591 32046 raft_consensus.cc:504] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badb62e50c344bf09319d802c15a8a8e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41885 } }
I20260807 03:07:13.649058 32046 leader_election.cc:302] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: badb62e50c344bf09319d802c15a8a8e; no voters:
I20260807 03:07:13.649605 32046 leader_election.cc:288] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:13.649742 32053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:13.650454 32053 raft_consensus.cc:686] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [term 1 LEADER]: Becoming Leader. State: Replica: badb62e50c344bf09319d802c15a8a8e, State: Running, Role: LEADER
I20260807 03:07:13.650960 32053 consensus_queue.cc:237] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badb62e50c344bf09319d802c15a8a8e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41885 } }
I20260807 03:07:13.651494 32046 sys_catalog.cc:564] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:13.653606 32055 sys_catalog.cc:455] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "badb62e50c344bf09319d802c15a8a8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badb62e50c344bf09319d802c15a8a8e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41885 } } }
I20260807 03:07:13.653998 32055 sys_catalog.cc:458] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:13.654575 32056 sys_catalog.cc:455] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader badb62e50c344bf09319d802c15a8a8e. Latest consensus state: current_term: 1 leader_uuid: "badb62e50c344bf09319d802c15a8a8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badb62e50c344bf09319d802c15a8a8e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 41885 } } }
I20260807 03:07:13.654935 32056 sys_catalog.cc:458] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:13.656430 32067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:13.661144 32067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:13.669615 32067 catalog_manager.cc:1347] Generated new cluster ID: cb1aec393c1a4bf1a0055db0a8d526a2
I20260807 03:07:13.669718 32067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:13.680785 32067 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:13.682551 32067 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:13.693696 32067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e: Generated new TSK 0
I20260807 03:07:13.694689 32067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:13.701517 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41885
--builtin_ntp_servers=127.5.122.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 03:07:14.025149 32086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:14.025417 32086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:14.025502 32086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:14.032722 32086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:14.032819 32086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:14.032920 32086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:07:14.041821 32086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:14.043058 32086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:14.044724 32086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:14.056880 32094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:14.056932 32096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:14.057476 32092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:14.057730 32086 server_base.cc:1034] running on GCE node
I20260807 03:07:14.058337 32086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:14.059207 32086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:14.060410 32086 hybrid_clock.cc:648] HybridClock initialized: now 1786072034060344 us; error 49 us; skew 500 ppm
I20260807 03:07:14.060698 32086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:14.062753 32086 webserver.cc:460] Webserver started at http://127.5.122.1:32979/ using document root <none> and password file <none>
I20260807 03:07:14.063397 32086 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:14.063494 32086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:14.063795 32086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:14.066558 32086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "b1127b3421e14823ab9282133c17c85d"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "d9cbd686ad94a08db28d709b63e50151"
server_key_iv: "78ba3dff17103a71e2c592483b0ec77d"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.067286 32086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "b1127b3421e14823ab9282133c17c85d"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "d9cbd686ad94a08db28d709b63e50151"
server_key_iv: "78ba3dff17103a71e2c592483b0ec77d"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.072028 32086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:07:14.075340 32106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.076510 32086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:07:14.076647 32086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "b1127b3421e14823ab9282133c17c85d"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "d9cbd686ad94a08db28d709b63e50151"
server_key_iv: "78ba3dff17103a71e2c592483b0ec77d"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.076839 32086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:14.100574 32086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:14.101379 32086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:14.101639 32086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:14.102634 32086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:14.104056 32086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:14.104149 32086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.104254 32086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:14.104326 32086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.135231 32086 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:39557
I20260807 03:07:14.135370 32268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:39557 every 8 connection(s)
I20260807 03:07:14.136708 32086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:07:14.139509 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 32086
I20260807 03:07:14.139709 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:07:14.140352 5608 external_mini_cluster.cc:1428] Setting key f3e1fcac87be8aa798a75ab149cf2b7b
I20260807 03:07:14.143548 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41885
--builtin_ntp_servers=127.5.122.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 03:07:14.149771 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:14.139232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:59079 (local address 127.5.122.62:41885)
0807 03:07:14.139633 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:14.139638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:14.141540 (+ 1902us) server_negotiation.cc:407] Connection header received
0807 03:07:14.143280 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:14.143286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:14.143374 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:14.143505 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:14.144530 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.145544 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.147049 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.147625 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.147761 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:14.148911 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:14.148918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:14.148923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:14.148946 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:14.149012 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:14.149030 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:14.149127 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:14.149298 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 03:07:14.149401 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":194,"threads_started":1}
I20260807 03:07:14.150534 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:14.139969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:41885 (local address 127.5.122.1:59079)
0807 03:07:14.140950 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:14.140997 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:14.143027 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:14.143667 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:14.143678 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:14.143796 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:14.144367 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:14.144382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.145671 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.145674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:14.146910 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:14.146921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.147808 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.147812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:14.147992 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:14.148616 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:14.148618 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:14.148654 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:14.148726 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:14.149121 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:14.149128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:14.149133 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:07:14.149257 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 03:07:14.149524 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":296,"threads_started":1}
I20260807 03:07:14.151897 32269 heartbeater.cc:347] Connected to a master server at 127.5.122.62:41885
I20260807 03:07:14.152184 32269 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:14.152964 32269 heartbeater.cc:511] Master 127.5.122.62:41885 requested a full tablet report, sending...
I20260807 03:07:14.154950 31995 ts_manager.cc:194] Registered new tserver with Master: b1127b3421e14823ab9282133c17c85d (127.5.122.1:39557)
I20260807 03:07:14.156514 31995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.1:59079
W20260807 03:07:14.465435 32273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:14.465687 32273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:14.465812 32273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:14.472867 32273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:14.472957 32273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:14.473109 32273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:07:14.482102 32273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:14.483300 32273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:14.484989 32273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:14.497253 32288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:14.497291 32291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:14.497594 32286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:14.498216 32273 server_base.cc:1034] running on GCE node
I20260807 03:07:14.498816 32273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:14.499725 32273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:14.500927 32273 hybrid_clock.cc:648] HybridClock initialized: now 1786072034500871 us; error 48 us; skew 500 ppm
I20260807 03:07:14.501221 32273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:14.503387 32273 webserver.cc:460] Webserver started at http://127.5.122.2:38965/ using document root <none> and password file <none>
I20260807 03:07:14.504029 32273 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:14.504128 32273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:14.504521 32273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:14.507368 32273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "bdafee0932994856af13bbe5003de3a2"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "07a66b7050c39e20dbc9f82200a3d1f5"
server_key_iv: "17380a16690d24c1dd3d9ff61d32ce63"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.508121 32273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "bdafee0932994856af13bbe5003de3a2"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "07a66b7050c39e20dbc9f82200a3d1f5"
server_key_iv: "17380a16690d24c1dd3d9ff61d32ce63"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.512969 32273 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 03:07:14.516295 32300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.517589 32273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 03:07:14.517755 32273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "bdafee0932994856af13bbe5003de3a2"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "07a66b7050c39e20dbc9f82200a3d1f5"
server_key_iv: "17380a16690d24c1dd3d9ff61d32ce63"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.517956 32273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:14.530867 32273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:14.531643 32273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:14.531895 32273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:14.532855 32273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:14.534304 32273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:14.534418 32273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.534526 32273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:14.534597 32273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.564380 32273 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:39743
I20260807 03:07:14.564429 32476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:39743 every 8 connection(s)
I20260807 03:07:14.565651 32273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:07:14.573527 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 32273
I20260807 03:07:14.573704 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:07:14.574131 5608 external_mini_cluster.cc:1428] Setting key 2d8c415a7ae9b40af1e3d2082a89fbdf
I20260807 03:07:14.577054 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41885
--builtin_ntp_servers=127.5.122.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 03:07:14.577910 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:14.568011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:38759 (local address 127.5.122.62:41885)
0807 03:07:14.568172 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:14.568177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:14.569479 (+ 1302us) server_negotiation.cc:407] Connection header received
0807 03:07:14.571245 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:14.571250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:14.571328 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:14.571417 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:14.572453 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.573172 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.575043 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.575671 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.575800 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:14.577068 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:14.577073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:14.577076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:14.577093 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:14.577124 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:14.577135 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:14.577206 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:14.577500 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 03:07:14.577598 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":53}
I20260807 03:07:14.578896 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:14.568474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:41885 (local address 127.5.122.2:38759)
0807 03:07:14.569326 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:14.569367 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:14.570994 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:14.571651 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:14.571659 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:14.571740 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:14.572284 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:14.572300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.573317 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.573321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:14.574859 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:14.574875 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:14.575878 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:14.575886 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:14.576148 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:14.576772 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:14.576775 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:14.576812 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:14.576885 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:14.577302 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:14.577311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:14.577317 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 03:07:14.577463 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 03:07:14.577827 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":268,"threads_started":1}
I20260807 03:07:14.580180 32477 heartbeater.cc:347] Connected to a master server at 127.5.122.62:41885
I20260807 03:07:14.580459 32477 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:14.581243 32477 heartbeater.cc:511] Master 127.5.122.62:41885 requested a full tablet report, sending...
I20260807 03:07:14.582680 31995 ts_manager.cc:194] Registered new tserver with Master: bdafee0932994856af13bbe5003de3a2 (127.5.122.2:39743)
I20260807 03:07:14.583669 31995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.2:38759
W20260807 03:07:14.900671 32481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:14.900930 32481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:14.901021 32481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:14.907919 32481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:14.908001 32481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:14.908109 32481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:07:14.916630 32481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:41885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:14.917723 32481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:14.919399 32481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:14.931520 32489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:14.931509 32488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:14.931655 32491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:14.932152 32481 server_base.cc:1034] running on GCE node
I20260807 03:07:14.932713 32481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:14.933559 32481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:14.934749 32481 hybrid_clock.cc:648] HybridClock initialized: now 1786072034934695 us; error 46 us; skew 500 ppm
I20260807 03:07:14.935040 32481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:14.937112 32481 webserver.cc:460] Webserver started at http://127.5.122.3:37471/ using document root <none> and password file <none>
I20260807 03:07:14.937729 32481 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:14.937827 32481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:14.938145 32481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:14.940883 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "8b940a253d5344209af6e1c2de9d9e7a"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "4ba26ad781d63b9dcc227538fc4464da"
server_key_iv: "22f8902d898a33b4e42d9f213f7b3a09"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.941623 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "8b940a253d5344209af6e1c2de9d9e7a"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "4ba26ad781d63b9dcc227538fc4464da"
server_key_iv: "22f8902d898a33b4e42d9f213f7b3a09"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.946300 32481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260807 03:07:14.949477 32499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.950742 32481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 03:07:14.950883 32481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "8b940a253d5344209af6e1c2de9d9e7a"
format_stamp: "Formatted at 2026-08-07 03:07:14 on dist-test-slave-fgb3"
server_key: "4ba26ad781d63b9dcc227538fc4464da"
server_key_iv: "22f8902d898a33b4e42d9f213f7b3a09"
server_key_version: "encryptionkey@0"
I20260807 03:07:14.951076 32481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:14.979894 32481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:14.980697 32481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:14.980964 32481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:14.981968 32481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:14.983345 32481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:14.983443 32481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:14.983548 32481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:14.983616 32481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:15.017509 32481 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:33663
I20260807 03:07:15.017560 32671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:33663 every 8 connection(s)
I20260807 03:07:15.018801 32481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:07:15.025760 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 32481
I20260807 03:07:15.025923 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:07:15.026372 5608 external_mini_cluster.cc:1428] Setting key 618840fdabfc11b7e6085f12d66e4ef0
I20260807 03:07:15.031034 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.021307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:33807 (local address 127.5.122.62:41885)
0807 03:07:15.021505 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.021510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.022866 (+ 1356us) server_negotiation.cc:407] Connection header received
0807 03:07:15.024570 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.024575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.024643 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.024728 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:15.025714 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.026640 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.028405 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.028925 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.029048 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.030190 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:15.030196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:15.030201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:15.030217 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:15.030250 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:15.030261 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:15.030363 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.030565 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.030657 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 03:07:15.031869 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.021808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:41885 (local address 127.5.122.3:33807)
0807 03:07:15.022725 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.022760 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:15.024310 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:15.024904 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:15.024911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:15.024990 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:15.025542 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.025559 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.026788 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.026792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.028278 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.028290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.029140 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.029148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.029346 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.029911 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:15.029913 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:15.029949 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:15.030019 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:15.030366 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:15.030377 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:15.030385 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 03:07:15.030539 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 03:07:15.030876 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":259,"threads_started":1}
I20260807 03:07:15.033008 32672 heartbeater.cc:347] Connected to a master server at 127.5.122.62:41885
I20260807 03:07:15.033269 32672 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:15.033960 32672 heartbeater.cc:511] Master 127.5.122.62:41885 requested a full tablet report, sending...
I20260807 03:07:15.035236 31995 ts_manager.cc:194] Registered new tserver with Master: 8b940a253d5344209af6e1c2de9d9e7a (127.5.122.3:33663)
I20260807 03:07:15.036010 31995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.3:33807
I20260807 03:07:15.040910 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:07:15.050112 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.044856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50082 (local address 127.5.122.62:41885)
0807 03:07:15.045057 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.045063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.045410 (+ 347us) server_negotiation.cc:407] Connection header received
0807 03:07:15.045535 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.045539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.045609 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.045682 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:15.046104 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.046792 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.048221 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.048576 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.048656 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.049566 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:15.049570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:15.049573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:15.049589 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:15.049612 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:15.049622 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:15.049678 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.049760 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.049829 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 03:07:15.053521 31995 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:07:15.060309 31995 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:07:15.092031 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.071451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39743 (local address 127.0.0.1:41158)
0807 03:07:15.083500 (+ 12049us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.083563 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:15.083768 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:15.085201 (+ 1433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:15.085210 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:15.085242 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:15.085572 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.085585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.087566 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.087572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.089348 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.089360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.091122 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.091128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.091337 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.091389 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 03:07:15.091519 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 03:07:15.091690 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11755,"thread_start_us":13410,"threads_started":1}
I20260807 03:07:15.092059 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.071649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.5.122.2:39743)
0807 03:07:15.084298 (+ 12649us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.084308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.084341 (+ 33us) server_negotiation.cc:407] Connection header received
0807 03:07:15.084448 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.084456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.084709 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.084901 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:15.085782 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.087356 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.089528 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.090818 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.090949 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.091172 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.091578 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.091742 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12499,"thread_start_us":12390,"threads_started":1}
I20260807 03:07:15.093431 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.071626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60604)
0807 03:07:15.084630 (+ 13004us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.084652 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:15.084778 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:15.086025 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:15.086031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:15.086059 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:15.086790 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.086806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.088297 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.088302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.090402 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.090415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.092408 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.092414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.092933 (+ 519us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.092965 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:07:15.093055 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 03:07:15.093168 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12881,"thread_start_us":11441,"threads_started":1}
I20260807 03:07:15.093649 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.072112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.5.122.1:39557)
0807 03:07:15.085483 (+ 13371us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.085492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.085520 (+ 28us) server_negotiation.cc:407] Connection header received
0807 03:07:15.085610 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.085618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.085825 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.086010 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:15.086980 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.088140 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.090628 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.091957 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.092076 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.092288 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.093144 (+ 856us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.093315 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13216,"thread_start_us":13088,"threads_started":1}
I20260807 03:07:15.095883 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.071958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:33663 (local address 127.0.0.1:49960)
0807 03:07:15.086617 (+ 14659us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.086638 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:15.086757 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:15.087566 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:15.087571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:15.087594 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:15.088020 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.088032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.089920 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.089926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.091798 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.091812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.095204 (+ 3392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.095210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.095388 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.095418 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 03:07:15.095511 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 03:07:15.095620 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14553,"thread_start_us":14411,"threads_started":1}
I20260807 03:07:15.096102 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.072539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.5.122.3:33663)
0807 03:07:15.085864 (+ 13325us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.085882 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.086875 (+ 993us) server_negotiation.cc:407] Connection header received
0807 03:07:15.086965 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.086972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.087150 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.087313 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:15.088238 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.089776 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.092009 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.094722 (+ 2713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.094843 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.095087 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.095574 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.095733 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13160,"thread_start_us":13011,"threads_started":1}
I20260807 03:07:15.096347 32377 tablet_service.cc:1467] Processing CreateTablet for tablet f8ff8644975943068da66ec9c6d2b5b4 (DEFAULT_TABLE table=test-table [id=9003c4846de444c3b9296527c3740809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:15.096961 32181 tablet_service.cc:1467] Processing CreateTablet for tablet f8ff8644975943068da66ec9c6d2b5b4 (DEFAULT_TABLE table=test-table [id=9003c4846de444c3b9296527c3740809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:15.097617 32377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ff8644975943068da66ec9c6d2b5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:15.098057 32181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ff8644975943068da66ec9c6d2b5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:15.099910 32576 tablet_service.cc:1467] Processing CreateTablet for tablet f8ff8644975943068da66ec9c6d2b5b4 (DEFAULT_TABLE table=test-table [id=9003c4846de444c3b9296527c3740809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:15.101181 32576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ff8644975943068da66ec9c6d2b5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:15.108667 32702 tablet_bootstrap.cc:492] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: Bootstrap starting.
I20260807 03:07:15.111505 32703 tablet_bootstrap.cc:492] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Bootstrap starting.
I20260807 03:07:15.112286 32702 tablet_bootstrap.cc:654] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:15.113481 32702 log.cc:824] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:15.114168 32704 tablet_bootstrap.cc:492] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Bootstrap starting.
I20260807 03:07:15.115274 32703 tablet_bootstrap.cc:654] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:15.116439 32703 log.cc:824] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:15.116895 32704 tablet_bootstrap.cc:654] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:15.117242 32702 tablet_bootstrap.cc:492] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: No bootstrap required, opened a new log
I20260807 03:07:15.117467 32702 ts_tablet_manager.cc:1397] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 03:07:15.118029 32704 log.cc:824] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:15.119378 32703 tablet_bootstrap.cc:492] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: No bootstrap required, opened a new log
I20260807 03:07:15.119635 32703 ts_tablet_manager.cc:1397] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 03:07:15.120364 32704 tablet_bootstrap.cc:492] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: No bootstrap required, opened a new log
I20260807 03:07:15.120661 32704 ts_tablet_manager.cc:1397] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 03:07:15.127540 32702 raft_consensus.cc:348] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.127966 32702 raft_consensus.cc:374] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:15.127835 32704 raft_consensus.cc:348] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.128062 32702 raft_consensus.cc:729] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1127b3421e14823ab9282133c17c85d, State: Initialized, Role: FOLLOWER
I20260807 03:07:15.128199 32704 raft_consensus.cc:374] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:15.128299 32704 raft_consensus.cc:729] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b940a253d5344209af6e1c2de9d9e7a, State: Initialized, Role: FOLLOWER
I20260807 03:07:15.128805 32704 consensus_queue.cc:260] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.128747 32702 consensus_queue.cc:260] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.129804 32672 heartbeater.cc:503] Master 127.5.122.62:41885 was elected leader, sending a full tablet report...
I20260807 03:07:15.129791 32703 raft_consensus.cc:348] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.130190 32703 raft_consensus.cc:374] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:15.130275 32703 raft_consensus.cc:729] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdafee0932994856af13bbe5003de3a2, State: Initialized, Role: FOLLOWER
I20260807 03:07:15.130470 32704 ts_tablet_manager.cc:1428] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Time spent starting tablet: real 0.010s user 0.002s sys 0.008s
I20260807 03:07:15.130950 32703 consensus_queue.cc:260] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.132134 32269 heartbeater.cc:503] Master 127.5.122.62:41885 was elected leader, sending a full tablet report...
I20260807 03:07:15.134022 32702 ts_tablet_manager.cc:1428] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260807 03:07:15.134501 32477 heartbeater.cc:503] Master 127.5.122.62:41885 was elected leader, sending a full tablet report...
I20260807 03:07:15.135361 32703 ts_tablet_manager.cc:1428] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260807 03:07:15.138676 32270 tablet.cc:2369] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:07:15.168934 32714 raft_consensus.cc:482] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:15.169229 32714 raft_consensus.cc:504] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.170907 32714 leader_election.cc:288] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdafee0932994856af13bbe5003de3a2 (127.5.122.2:39743), 8b940a253d5344209af6e1c2de9d9e7a (127.5.122.3:33663)
I20260807 03:07:15.178231 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.171423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:42053 (local address 127.5.122.2:39743)
0807 03:07:15.171626 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.171632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.172297 (+ 665us) server_negotiation.cc:407] Connection header received
0807 03:07:15.172504 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.172509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.172602 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.172700 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:15.173416 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.174537 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.176634 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.177378 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.177490 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.177581 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.177837 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.177972 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 03:07:15.178297 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.171478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:60189 (local address 127.5.122.3:33663)
0807 03:07:15.171652 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.171658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.172353 (+ 695us) server_negotiation.cc:407] Connection header received
0807 03:07:15.172536 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.172541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.172609 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.172700 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:15.173521 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.174600 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.176659 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.177425 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.177540 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.177636 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.177923 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.178064 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 03:07:15.178414 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.171141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:39743 (local address 127.5.122.1:42053)
0807 03:07:15.172171 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.172193 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:15.172366 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:15.172723 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:15.172728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:15.172784 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:15.173272 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.173290 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.174651 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.174657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.176432 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.176453 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.177562 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.177568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.177695 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.177718 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 03:07:15.177799 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 03:07:15.177924 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":107,"spinlock_wait_cycles":28416,"thread_start_us":663,"threads_started":1}
I20260807 03:07:15.178558 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.171234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:33663 (local address 127.5.122.1:60189)
0807 03:07:15.172244 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.172258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:15.172420 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:15.172723 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:15.172728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:15.172758 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:15.173339 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.173351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.174712 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.174717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.176446 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:15.176462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.177619 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.177625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:15.177774 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.177801 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 03:07:15.177877 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 03:07:15.178006 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":87,"spinlock_wait_cycles":24448,"thread_start_us":586,"threads_started":1}
I20260807 03:07:15.179674 32608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff8644975943068da66ec9c6d2b5b4" candidate_uuid: "b1127b3421e14823ab9282133c17c85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b940a253d5344209af6e1c2de9d9e7a" is_pre_election: true
I20260807 03:07:15.179687 32406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff8644975943068da66ec9c6d2b5b4" candidate_uuid: "b1127b3421e14823ab9282133c17c85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdafee0932994856af13bbe5003de3a2" is_pre_election: true
I20260807 03:07:15.180261 32608 raft_consensus.cc:2454] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1127b3421e14823ab9282133c17c85d in term 0.
I20260807 03:07:15.180292 32406 raft_consensus.cc:2454] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1127b3421e14823ab9282133c17c85d in term 0.
I20260807 03:07:15.180907 32107 leader_election.cc:302] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8b940a253d5344209af6e1c2de9d9e7a, b1127b3421e14823ab9282133c17c85d, bdafee0932994856af13bbe5003de3a2; no voters:
I20260807 03:07:15.181259 32714 raft_consensus.cc:2781] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:07:15.181406 32714 raft_consensus.cc:482] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:07:15.181504 32714 raft_consensus.cc:3037] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:15.183728 32714 raft_consensus.cc:504] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.184332 32714 leader_election.cc:288] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [CANDIDATE]: Term 1 election: Requested vote from peers bdafee0932994856af13bbe5003de3a2 (127.5.122.2:39743), 8b940a253d5344209af6e1c2de9d9e7a (127.5.122.3:33663)
I20260807 03:07:15.184791 32406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff8644975943068da66ec9c6d2b5b4" candidate_uuid: "b1127b3421e14823ab9282133c17c85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdafee0932994856af13bbe5003de3a2"
I20260807 03:07:15.184831 32608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ff8644975943068da66ec9c6d2b5b4" candidate_uuid: "b1127b3421e14823ab9282133c17c85d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b940a253d5344209af6e1c2de9d9e7a"
I20260807 03:07:15.185019 32608 raft_consensus.cc:3037] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:15.185019 32406 raft_consensus.cc:3037] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:15.188247 32608 raft_consensus.cc:2454] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1127b3421e14823ab9282133c17c85d in term 1.
I20260807 03:07:15.188247 32406 raft_consensus.cc:2454] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1127b3421e14823ab9282133c17c85d in term 1.
I20260807 03:07:15.188695 32111 leader_election.cc:302] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8b940a253d5344209af6e1c2de9d9e7a, b1127b3421e14823ab9282133c17c85d, bdafee0932994856af13bbe5003de3a2; no voters:
I20260807 03:07:15.189033 32714 raft_consensus.cc:2781] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:15.189481 32714 raft_consensus.cc:686] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 1 LEADER]: Becoming Leader. State: Replica: b1127b3421e14823ab9282133c17c85d, State: Running, Role: LEADER
I20260807 03:07:15.189944 32714 consensus_queue.cc:237] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } }
I20260807 03:07:15.194743 31994 catalog_manager.cc:5495] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1127b3421e14823ab9282133c17c85d (127.5.122.1). New cstate: current_term: 1 leader_uuid: "b1127b3421e14823ab9282133c17c85d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1127b3421e14823ab9282133c17c85d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 39557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 } health_report { overall_health: UNKNOWN } } }
I20260807 03:07:15.215536 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.210575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.5.122.1:39557)
0807 03:07:15.210754 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.210759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.210848 (+ 89us) server_negotiation.cc:407] Connection header received
0807 03:07:15.211035 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.211039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.211129 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.211257 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:07:15.211555 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.212242 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.213610 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.214006 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.214091 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.214581 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:07:15.215076 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:07:15.215185 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.215257 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.215322 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:07:15.223703 32406 raft_consensus.cc:1261] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 1 FOLLOWER]: Refusing update from remote peer b1127b3421e14823ab9282133c17c85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:15.223702 32608 raft_consensus.cc:1261] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 1 FOLLOWER]: Refusing update from remote peer b1127b3421e14823ab9282133c17c85d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:15.224615 32718 consensus_queue.cc:1035] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdafee0932994856af13bbe5003de3a2" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:15.224948 32714 consensus_queue.cc:1035] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b940a253d5344209af6e1c2de9d9e7a" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:15.248078 31994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:15.248332 31994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:15.251593 31994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P badb62e50c344bf09319d802c15a8a8e: Sending DeleteTablet for 3 replicas of tablet f8ff8644975943068da66ec9c6d2b5b4
I20260807 03:07:15.252812 32181 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ff8644975943068da66ec9c6d2b5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:15 UTC) from {username='slave'} at 127.0.0.1:60604
I20260807 03:07:15.252810 32377 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ff8644975943068da66ec9c6d2b5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:15 UTC) from {username='slave'} at 127.0.0.1:41158
I20260807 03:07:15.253142 32576 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ff8644975943068da66ec9c6d2b5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:15 UTC) from {username='slave'} at 127.0.0.1:49960
I20260807 03:07:15.254746 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 32086
I20260807 03:07:15.266471 32738 tablet_replica.cc:330] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: stopping tablet replica
I20260807 03:07:15.266763 32734 tablet_replica.cc:330] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: stopping tablet replica
I20260807 03:07:15.266959 32738 raft_consensus.cc:2229] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:15.267141 32734 raft_consensus.cc:2229] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:07:15.267509 32738 raft_consensus.cc:2258] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:15.267769 32734 raft_consensus.cc:2258] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:15.268404 32736 tablet_replica.cc:330] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: stopping tablet replica
I20260807 03:07:15.269187 32736 raft_consensus.cc:2229] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:15.269695 32736 raft_consensus.cc:2258] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:15.270236 32738 ts_tablet_manager.cc:1905] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:15.271202 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.256246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.5.122.1:39557)
0807 03:07:15.256426 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.256432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.263132 (+ 6700us) server_negotiation.cc:407] Connection header received
0807 03:07:15.263328 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.263334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.263438 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.263549 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:15.264236 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.265487 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.267358 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.267860 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.267981 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.269370 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:15.269381 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:15.269398 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:15.270225 (+ 827us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:15.270285 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:15.270331 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:15.270424 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.270733 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.270832 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 03:07:15.271924 32734 ts_tablet_manager.cc:1905] T f8ff8644975943068da66ec9c6d2b5b4 P b1127b3421e14823ab9282133c17c85d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:15.272360 32264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:15.274837 32736 ts_tablet_manager.cc:1905] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:15.278549 32738 ts_tablet_manager.cc:1918] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:15.278752 32738 log.cc:1196] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/f8ff8644975943068da66ec9c6d2b5b4
I20260807 03:07:15.279410 32738 ts_tablet_manager.cc:1939] T f8ff8644975943068da66ec9c6d2b5b4 P 8b940a253d5344209af6e1c2de9d9e7a: Deleting consensus metadata
I20260807 03:07:15.281172 31977 catalog_manager.cc:4841] TS 8b940a253d5344209af6e1c2de9d9e7a (127.5.122.3:33663): tablet f8ff8644975943068da66ec9c6d2b5b4 (table test-table [id=9003c4846de444c3b9296527c3740809]) successfully deleted
I20260807 03:07:15.281808 32736 ts_tablet_manager.cc:1918] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:15.281925 32736 log.cc:1196] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/f8ff8644975943068da66ec9c6d2b5b4
I20260807 03:07:15.282359 32736 ts_tablet_manager.cc:1939] T f8ff8644975943068da66ec9c6d2b5b4 P bdafee0932994856af13bbe5003de3a2: Deleting consensus metadata
I20260807 03:07:15.283666 31973 catalog_manager.cc:4841] TS bdafee0932994856af13bbe5003de3a2 (127.5.122.2:39743): tablet f8ff8644975943068da66ec9c6d2b5b4 (table test-table [id=9003c4846de444c3b9296527c3740809]) successfully deleted
I20260807 03:07:15.370837 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 32086
W20260807 03:07:15.392374 31975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.1:39557 (error 108)
I20260807 03:07:15.392558 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 32273
I20260807 03:07:15.394239 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.393773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60626)
0807 03:07:15.393949 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.394066 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 03:07:15.394598 31975 catalog_manager.cc:4584] TS b1127b3421e14823ab9282133c17c85d (127.5.122.1:39557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8ff8644975943068da66ec9c6d2b5b4: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
I20260807 03:07:15.399309 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.393331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.5.122.2:39743)
0807 03:07:15.393501 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.393506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.393640 (+ 134us) server_negotiation.cc:407] Connection header received
0807 03:07:15.393824 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.393829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.393958 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.394054 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:15.394554 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.395411 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.396594 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.396986 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.397078 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.397947 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:15.397956 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:15.397969 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:15.398655 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:15.398702 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:15.398723 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:15.398823 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.398907 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.398998 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:07:15.399899 32472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:15.448258 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.447781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60638)
0807 03:07:15.447993 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.448088 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 03:07:15.449272 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.448935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60652)
0807 03:07:15.449077 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.449147 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 03:07:15.486922 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 32273
I20260807 03:07:15.508412 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 32481
I20260807 03:07:15.509999 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.509591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60658)
0807 03:07:15.509753 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.509825 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 03:07:15.510919 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.510602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60662)
0807 03:07:15.510737 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.510803 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 03:07:15.514963 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.509152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.5.122.3:33663)
0807 03:07:15.509361 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:15.509367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:15.509387 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:07:15.509541 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:15.509545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:15.509642 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:15.509730 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:15.510157 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.510924 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.512294 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:15.512692 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:15.512787 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:15.513645 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:15.513652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:15.513666 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:15.514237 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:15.514282 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:15.514306 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:15.514431 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:15.514616 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 03:07:15.514712 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 03:07:15.515517 32667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:15.591471 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.591020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60668)
0807 03:07:15.591204 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.591291 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 03:07:15.592563 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:15.592116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:39557 (local address 127.0.0.1:60678)
0807 03:07:15.592302 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:15.592409 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 03:07:15.603616 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 32481
I20260807 03:07:15.626130 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 31952
I20260807 03:07:15.627174 32041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:15.733830 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 31952
2026-08-07T03:07:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2639 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:15 dist-test-slave-fgb3 krb5kdc[32752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:15 dist-test-slave-fgb3 krb5kdc[32752](info): set up 2 sockets
Aug 07 03:07:15 dist-test-slave-fgb3 krb5kdc[32752](info): commencing operation
krb5kdc: starting...
W20260807 03:07:16.921998 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:17 dist-test-slave-fgb3 krb5kdc[32752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:07:17.173437 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:36835
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:36835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:17.486802 306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:17.487076 306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:17.487206 306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:17.494115 306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:17.494221 306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:17.494299 306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:17.494392 306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:17.494459 306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:17.504195 306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:36835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:36835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:17.505404 306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:17.506690 306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:17.517737 314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:17.517696 318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:17.517711 316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:17.518661 306 server_base.cc:1034] running on GCE node
I20260807 03:07:17.519363 306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:17.520619 306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:17.521823 306 hybrid_clock.cc:648] HybridClock initialized: now 1786072037521770 us; error 36 us; skew 500 ppm
I20260807 03:07:17.522148 306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:17 dist-test-slave-fgb3 krb5kdc[32752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072037, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:17.524969 306 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:07:17.526295 306 webserver.cc:460] Webserver started at http://127.5.122.62:36593/ using document root <none> and password file <none>
I20260807 03:07:17.526962 306 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:17.527078 306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:17.527365 306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:17.530153 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "c0b775fb3f34436ba5abaa99c9b594bd"
format_stamp: "Formatted at 2026-08-07 03:07:17 on dist-test-slave-fgb3"
server_key: "815632c12a2e1003d53aa3cf9becac5c"
server_key_iv: "4b035a69df5509a9007f9f6e38d55f33"
server_key_version: "encryptionkey@0"
I20260807 03:07:17.530896 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "c0b775fb3f34436ba5abaa99c9b594bd"
format_stamp: "Formatted at 2026-08-07 03:07:17 on dist-test-slave-fgb3"
server_key: "815632c12a2e1003d53aa3cf9becac5c"
server_key_iv: "4b035a69df5509a9007f9f6e38d55f33"
server_key_version: "encryptionkey@0"
I20260807 03:07:17.535595 306 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 03:07:17.538699 328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:17.539824 306 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:07:17.539980 306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "c0b775fb3f34436ba5abaa99c9b594bd"
format_stamp: "Formatted at 2026-08-07 03:07:17 on dist-test-slave-fgb3"
server_key: "815632c12a2e1003d53aa3cf9becac5c"
server_key_iv: "4b035a69df5509a9007f9f6e38d55f33"
server_key_version: "encryptionkey@0"
I20260807 03:07:17.540143 306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:17.553732 306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:17.557179 306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:17.557415 306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:17.574138 306 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:36835
I20260807 03:07:17.574180 403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:36835 every 8 connection(s)
I20260807 03:07:17.575488 306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:17.579094 404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:17.584442 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 306
I20260807 03:07:17.584626 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:17.585179 5608 external_mini_cluster.cc:1428] Setting key ab7c18eb00043a29ff1089e5b1c68676
I20260807 03:07:17.590101 404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd: Bootstrap starting.
I20260807 03:07:17.593683 404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:17.594928 404 log.cc:824] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:17.598315 404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd: No bootstrap required, opened a new log
Aug 07 03:07:17 dist-test-slave-fgb3 krb5kdc[32752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:17.608485 404 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36835 } }
I20260807 03:07:17.608946 404 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:17.609055 404 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b775fb3f34436ba5abaa99c9b594bd, State: Initialized, Role: FOLLOWER
I20260807 03:07:17.609570 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:17.587057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.5.122.62:36835)
0807 03:07:17.587881 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:17.587896 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:17.587939 (+ 43us) server_negotiation.cc:407] Connection header received
0807 03:07:17.589626 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:17.589648 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:17.590015 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:17.590374 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:17.592111 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:17.593377 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:17.595223 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:17.596220 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:17.596366 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:17.601119 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:17.601130 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:17.601150 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:17.601196 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:17.605084 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:17.605745 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:17.605753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:17.605762 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:17.605993 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:17.606365 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:17.606370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:17.606373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:17.607128 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:17.607420 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:17.607785 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 03:07:17.608222 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":394,"threads_started":1}
I20260807 03:07:17.609747 404 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36835 } }
I20260807 03:07:17.610037 404 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:17.610177 404 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:17.610390 404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:17.613160 404 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36835 } }
I20260807 03:07:17.613610 404 leader_election.cc:302] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0b775fb3f34436ba5abaa99c9b594bd; no voters:
I20260807 03:07:17.614131 404 leader_election.cc:288] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:17.614245 411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:17.614892 411 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [term 1 LEADER]: Becoming Leader. State: Replica: c0b775fb3f34436ba5abaa99c9b594bd, State: Running, Role: LEADER
I20260807 03:07:17.615439 411 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36835 } }
I20260807 03:07:17.616040 404 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:17.618635 414 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0b775fb3f34436ba5abaa99c9b594bd. Latest consensus state: current_term: 1 leader_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36835 } } }
I20260807 03:07:17.618646 413 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b775fb3f34436ba5abaa99c9b594bd" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 36835 } } }
I20260807 03:07:17.619079 413 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:17.619078 414 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:17.620263 424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:17.624369 424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:17.632033 424 catalog_manager.cc:1347] Generated new cluster ID: ad15f61dd894408d92426dbe54a2983e
I20260807 03:07:17.632133 424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:17.653807 424 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:17.655637 424 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:17.665591 424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd: Generated new TSK 0
I20260807 03:07:17.666431 424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:07:17.733387 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:36835
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:18.049660 441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:18.049916 441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:18.050019 441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:18.056921 441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:18.057013 441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:18.057132 441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:07:18.066138 441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:36835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:18.067274 441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:18.068496 441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:18.080533 448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:18.080428 450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:18.080593 457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:18.081329 441 server_base.cc:1034] running on GCE node
I20260807 03:07:18.081844 441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:18.082705 441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:18.083894 441 hybrid_clock.cc:648] HybridClock initialized: now 1786072038083856 us; error 28 us; skew 500 ppm
I20260807 03:07:18.084172 441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:18 dist-test-slave-fgb3 krb5kdc[32752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:18.087131 441 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:07:18.088452 441 webserver.cc:460] Webserver started at http://127.5.122.1:40271/ using document root <none> and password file <none>
I20260807 03:07:18.089058 441 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:18.089166 441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:18.089457 441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:18.092159 441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "ae830da71b6e4b12bb9ef6123daee264"
format_stamp: "Formatted at 2026-08-07 03:07:18 on dist-test-slave-fgb3"
server_key: "5e6210ab86a781673cf58377fbf9b61c"
server_key_iv: "bb49c62f072fe65e865ffaff78ce234c"
server_key_version: "encryptionkey@0"
I20260807 03:07:18.092903 441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "ae830da71b6e4b12bb9ef6123daee264"
format_stamp: "Formatted at 2026-08-07 03:07:18 on dist-test-slave-fgb3"
server_key: "5e6210ab86a781673cf58377fbf9b61c"
server_key_iv: "bb49c62f072fe65e865ffaff78ce234c"
server_key_version: "encryptionkey@0"
I20260807 03:07:18.097654 441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 03:07:18.100862 466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:18.102011 441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:07:18.102145 441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "ae830da71b6e4b12bb9ef6123daee264"
format_stamp: "Formatted at 2026-08-07 03:07:18 on dist-test-slave-fgb3"
server_key: "5e6210ab86a781673cf58377fbf9b61c"
server_key_iv: "bb49c62f072fe65e865ffaff78ce234c"
server_key_version: "encryptionkey@0"
I20260807 03:07:18.102360 441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:18.119371 441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:18.122953 441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:18.123172 441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:18.123948 441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:18.125131 441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:18.125204 441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:18.125305 441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:18.125365 441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:18.154670 441 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:46197
I20260807 03:07:18.154731 644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:46197 every 8 connection(s)
I20260807 03:07:18.155824 441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:07:18.157222 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 441
I20260807 03:07:18.157407 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:07:18.157864 5608 external_mini_cluster.cc:1428] Setting key 74483a81ac8dab4d16dfa95dd1d39c36
Aug 07 03:07:18 dist-test-slave-fgb3 krb5kdc[32752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:18.175932 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:18.158538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:41515 (local address 127.5.122.62:36835)
0807 03:07:18.159007 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:18.159013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:18.159727 (+ 714us) server_negotiation.cc:407] Connection header received
0807 03:07:18.162027 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:18.162031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:18.162093 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:18.162176 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:18.163880 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.164595 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.166334 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.166718 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.166837 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:18.170086 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:18.170092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:18.170097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:18.170133 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:18.172935 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:18.173767 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:18.173771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:18.173773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:18.173880 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:18.174354 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:18.174359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:18.174362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:18.174689 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:18.174828 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:18.175387 (+ 559us) server_negotiation.cc:299] Negotiation successful
0807 03:07:18.175601 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":241,"threads_started":1}
I20260807 03:07:18.176529 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:18.158751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:36835 (local address 127.5.122.1:41515)
0807 03:07:18.159565 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:18.159622 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:18.161761 (+ 2139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:18.162440 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:18.162450 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:18.163073 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:18.163732 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:18.163754 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.164765 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.164769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:18.166171 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:18.166185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.166931 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.166939 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:18.167135 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:18.167709 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:18.167735 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:18.169871 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:18.173089 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:18.173097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:18.173111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:18.173670 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:18.173972 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:18.173977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:18.173981 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:18.174212 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:18.174798 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:18.174805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:18.175071 (+ 266us) client_negotiation.cc:769] Sending connection context
0807 03:07:18.175284 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 03:07:18.175613 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":324,"threads_started":1}
I20260807 03:07:18.177824 646 heartbeater.cc:347] Connected to a master server at 127.5.122.62:36835
I20260807 03:07:18.178114 646 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:18.178882 646 heartbeater.cc:511] Master 127.5.122.62:36835 requested a full tablet report, sending...
I20260807 03:07:18.180930 355 ts_manager.cc:194] Registered new tserver with Master: ae830da71b6e4b12bb9ef6123daee264 (127.5.122.1:46197)
I20260807 03:07:18.182272 355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:41515
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:07:18.221516 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:36835
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:18.537232 654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:18.537484 654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:18.537576 654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:18.544677 654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:18.544780 654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:18.544890 654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:07:18.557155 654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:36835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:18.558377 654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:18.559624 654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:18.571591 663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:18.571630 664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:18.571655 668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:18.572742 654 server_base.cc:1034] running on GCE node
I20260807 03:07:18.573235 654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:18.574138 654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:18.575345 654 hybrid_clock.cc:648] HybridClock initialized: now 1786072038575289 us; error 49 us; skew 500 ppm
I20260807 03:07:18.575641 654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:18 dist-test-slave-fgb3 krb5kdc[32752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:18.581605 654 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:07:18.583338 654 webserver.cc:460] Webserver started at http://127.5.122.2:38631/ using document root <none> and password file <none>
I20260807 03:07:18.584141 654 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:18.584286 654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:18.584654 654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:18.588541 654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "a1aaecfe640f40be9751f1280f6233fb"
format_stamp: "Formatted at 2026-08-07 03:07:18 on dist-test-slave-fgb3"
server_key: "c1aa80f4866272bc61387efc9ebebc9a"
server_key_iv: "a1b2d9b869cd5dd563bf93e0cba6875d"
server_key_version: "encryptionkey@0"
I20260807 03:07:18.589529 654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "a1aaecfe640f40be9751f1280f6233fb"
format_stamp: "Formatted at 2026-08-07 03:07:18 on dist-test-slave-fgb3"
server_key: "c1aa80f4866272bc61387efc9ebebc9a"
server_key_iv: "a1b2d9b869cd5dd563bf93e0cba6875d"
server_key_version: "encryptionkey@0"
I20260807 03:07:18.594739 654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 03:07:18.597950 677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:18.599152 654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:07:18.599318 654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "a1aaecfe640f40be9751f1280f6233fb"
format_stamp: "Formatted at 2026-08-07 03:07:18 on dist-test-slave-fgb3"
server_key: "c1aa80f4866272bc61387efc9ebebc9a"
server_key_iv: "a1b2d9b869cd5dd563bf93e0cba6875d"
server_key_version: "encryptionkey@0"
I20260807 03:07:18.599498 654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:18.609783 654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:18.613363 654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:18.613636 654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:18.614554 654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:18.615828 654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:18.615939 654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:18.616067 654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:18.616127 654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:18.648072 654 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:34183
I20260807 03:07:18.648102 842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:34183 every 8 connection(s)
I20260807 03:07:18.649497 654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:07:18.656800 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 654
I20260807 03:07:18.656982 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:07:18.657513 5608 external_mini_cluster.cc:1428] Setting key eb80aadeac4858964b1254d6b49496b0
Aug 07 03:07:18 dist-test-slave-fgb3 krb5kdc[32752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:18.669196 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:18.651881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:41163 (local address 127.5.122.62:36835)
0807 03:07:18.652042 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:18.652046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:18.653109 (+ 1063us) server_negotiation.cc:407] Connection header received
0807 03:07:18.654917 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:18.654923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:18.655010 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:18.655095 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:18.656489 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.657254 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.658999 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.659532 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.659663 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:18.663080 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:18.663087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:18.663093 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:18.663136 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:18.666168 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:18.666871 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:18.666877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:18.666881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:18.667051 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:18.667432 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:18.667437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:18.667440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:18.667805 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:18.667960 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:18.668588 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 03:07:18.668826 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:07:18.669881 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:18.652290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:36835 (local address 127.5.122.2:41163)
0807 03:07:18.652960 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:18.652997 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:18.654668 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:18.655325 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:18.655333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:18.655811 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:18.656321 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:18.656337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.657396 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.657399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:18.658858 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:18.658878 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:18.659737 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:18.659742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:18.659937 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:18.660583 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:18.660604 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:18.662845 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:18.666300 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:18.666306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:18.666341 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:18.666736 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:18.667147 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:18.667150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:18.667152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:18.667324 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:18.667919 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:18.667924 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:18.668217 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 03:07:18.668431 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 03:07:18.668790 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":277,"threads_started":1}
I20260807 03:07:18.671195 846 heartbeater.cc:347] Connected to a master server at 127.5.122.62:36835
I20260807 03:07:18.671482 846 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:18.672118 846 heartbeater.cc:511] Master 127.5.122.62:36835 requested a full tablet report, sending...
I20260807 03:07:18.673411 355 ts_manager.cc:194] Registered new tserver with Master: a1aaecfe640f40be9751f1280f6233fb (127.5.122.2:34183)
I20260807 03:07:18.674177 355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:41163
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:07:18.720897 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:36835
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:19.042101 853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:19.042395 853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:19.042531 853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:19.049746 853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:19.049849 853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:19.050014 853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:07:19.059355 853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:36835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:19.060590 853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:19.061872 853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:19.074131 861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:19.074226 864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:19.074183 868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:19.075358 853 server_base.cc:1034] running on GCE node
I20260807 03:07:19.075930 853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:19.076781 853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:19.077998 853 hybrid_clock.cc:648] HybridClock initialized: now 1786072039077926 us; error 54 us; skew 500 ppm
I20260807 03:07:19.078286 853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:19 dist-test-slave-fgb3 krb5kdc[32752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072039, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:19.081264 853 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:07:19.082661 853 webserver.cc:460] Webserver started at http://127.5.122.3:36009/ using document root <none> and password file <none>
I20260807 03:07:19.083312 853 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:19.083432 853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:19.083716 853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:19.086531 853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "89f232551f4c4e32956df8750fa33ead"
format_stamp: "Formatted at 2026-08-07 03:07:19 on dist-test-slave-fgb3"
server_key: "4644787ac7a53aaa235dea68e99d697b"
server_key_iv: "162e73f31d18d38a3f557b5a1027307e"
server_key_version: "encryptionkey@0"
I20260807 03:07:19.087287 853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "89f232551f4c4e32956df8750fa33ead"
format_stamp: "Formatted at 2026-08-07 03:07:19 on dist-test-slave-fgb3"
server_key: "4644787ac7a53aaa235dea68e99d697b"
server_key_iv: "162e73f31d18d38a3f557b5a1027307e"
server_key_version: "encryptionkey@0"
I20260807 03:07:19.092118 853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:07:19.095343 877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:19.096513 853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:07:19.096662 853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "89f232551f4c4e32956df8750fa33ead"
format_stamp: "Formatted at 2026-08-07 03:07:19 on dist-test-slave-fgb3"
server_key: "4644787ac7a53aaa235dea68e99d697b"
server_key_iv: "162e73f31d18d38a3f557b5a1027307e"
server_key_version: "encryptionkey@0"
I20260807 03:07:19.096829 853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:19.109073 853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:19.112658 853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:19.112919 853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:19.113759 853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:19.115031 853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:19.115109 853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:19.115229 853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:19.115286 853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:19.145686 853 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:43855
I20260807 03:07:19.145736 1052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:43855 every 8 connection(s)
I20260807 03:07:19.146901 853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:07:19.156330 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 853
I20260807 03:07:19.156533 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:07:19.157069 5608 external_mini_cluster.cc:1428] Setting key 6c6e5250ed8f10800977c042c3b74351
Aug 07 03:07:19 dist-test-slave-fgb3 krb5kdc[32752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072039, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:19.165691 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.149221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53811 (local address 127.5.122.62:36835)
0807 03:07:19.149410 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.149414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.150612 (+ 1198us) server_negotiation.cc:407] Connection header received
0807 03:07:19.152467 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.152473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.152552 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.152639 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:19.154067 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.154939 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.156425 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.156911 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.157022 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.160773 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:19.160778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:19.160782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:19.160813 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:19.163317 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.163941 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.163945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.163947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.164058 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.164399 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.164402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.164404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.164643 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:19.164749 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.165304 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.165460 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 03:07:19.166661 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.149717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:36835 (local address 127.5.122.3:53811)
0807 03:07:19.150455 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:19.150496 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:19.152200 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:19.152870 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:19.152878 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:19.153367 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:19.153901 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.153918 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.155092 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.155096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.156273 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.156285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.157131 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.157140 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.157399 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.158117 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:19.158139 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:19.160581 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:19.163444 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:19.163451 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:19.163461 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:19.163857 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:19.164133 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:19.164136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:19.164139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:19.164321 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:19.164730 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:19.164735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:19.165025 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 03:07:19.165237 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 03:07:19.165593 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":269,"threads_started":1}
I20260807 03:07:19.167927 1053 heartbeater.cc:347] Connected to a master server at 127.5.122.62:36835
I20260807 03:07:19.168200 1053 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:19.168880 1053 heartbeater.cc:511] Master 127.5.122.62:36835 requested a full tablet report, sending...
I20260807 03:07:19.170199 355 ts_manager.cc:194] Registered new tserver with Master: 89f232551f4c4e32956df8750fa33ead (127.5.122.3:43855)
I20260807 03:07:19.171190 355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:53811
I20260807 03:07:19.172499 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:07:19.185393 646 heartbeater.cc:503] Master 127.5.122.62:36835 was elected leader, sending a full tablet report...
I20260807 03:07:19.189031 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.176349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.5.122.62:36835)
0807 03:07:19.176507 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.176511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.176936 (+ 425us) server_negotiation.cc:407] Connection header received
0807 03:07:19.177061 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.177064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.177118 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.177193 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:19.178610 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.179324 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.180603 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.181014 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.181100 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.183862 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:19.183867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:19.183870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:19.183900 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:19.186513 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.187370 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.187374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.187376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.187504 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.187843 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.187846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.187848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.188125 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:19.188321 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.188588 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.188714 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:07:19.192610 355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:07:19.199383 355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:07:19.222935 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.213164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.5.122.1:46197)
0807 03:07:19.214143 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.214150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.214174 (+ 24us) server_negotiation.cc:407] Connection header received
0807 03:07:19.214261 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.214267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.214509 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.215449 (+ 940us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:19.215694 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.217222 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.220638 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.221557 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.221673 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.221903 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.222434 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.222603 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":663,"threads_started":1}
I20260807 03:07:19.223014 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.212162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:46197 (local address 127.0.0.1:47390)
0807 03:07:19.213125 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:19.213142 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:19.213249 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:19.214722 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:19.214727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:19.214749 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:19.215244 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.215255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.217377 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.217383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.220494 (+ 3111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.220505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.222062 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.222069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.222226 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.222258 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:07:19.222385 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 03:07:19.222578 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":1195,"thread_start_us":690,"threads_started":1}
I20260807 03:07:19.223788 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.211686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.5.122.3:43855)
0807 03:07:19.213791 (+ 2105us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.213800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.213835 (+ 35us) server_negotiation.cc:407] Connection header received
0807 03:07:19.213940 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.213947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.214187 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.215220 (+ 1033us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:19.215507 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.217222 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.220121 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.221534 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.221658 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.223218 (+ 1560us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.223301 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.223458 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1951,"thread_start_us":1807,"threads_started":1}
I20260807 03:07:19.223848 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.211379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:43855 (local address 127.0.0.1:49766)
0807 03:07:19.212218 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:19.212268 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:19.212486 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:19.214459 (+ 1973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:19.214467 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:19.214503 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:19.214993 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.215006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.217379 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.217383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.219976 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.219992 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.221727 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.221733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.221948 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.222006 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 03:07:19.222149 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 03:07:19.223532 (+ 1383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":1198,"thread_start_us":360,"threads_started":1}
I20260807 03:07:19.226949 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.213280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59470 (local address 127.5.122.2:34183)
0807 03:07:19.219079 (+ 5799us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.219088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.219113 (+ 25us) server_negotiation.cc:407] Connection header received
0807 03:07:19.219202 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.219209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.219414 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.219598 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:19.220216 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.221711 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.224205 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.225349 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.225501 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.225692 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.226462 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.226633 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5640,"thread_start_us":5503,"threads_started":1}
I20260807 03:07:19.227043 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.212566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:34183 (local address 127.0.0.1:59470)
0807 03:07:19.218447 (+ 5881us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:19.218464 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:19.218578 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:19.219612 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:19.219617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:19.219643 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:19.220041 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.220055 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.221885 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.221890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.224017 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.224033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.225553 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.225560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.225725 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.225757 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:07:19.225851 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 03:07:19.225974 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5723,"mutex_wait_us":3,"thread_start_us":5571,"threads_started":1}
I20260807 03:07:19.227891 953 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec82229b8424ee5a7295f701202132d (DEFAULT_TABLE table=test-table [id=9df24bda04b14e2bb974e716e066c9d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:19.227907 535 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec82229b8424ee5a7295f701202132d (DEFAULT_TABLE table=test-table [id=9df24bda04b14e2bb974e716e066c9d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:19.229120 953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec82229b8424ee5a7295f701202132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:19.229209 535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec82229b8424ee5a7295f701202132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:19.231885 748 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec82229b8424ee5a7295f701202132d (DEFAULT_TABLE table=test-table [id=9df24bda04b14e2bb974e716e066c9d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:19.233225 748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec82229b8424ee5a7295f701202132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:19.240283 1085 tablet_bootstrap.cc:492] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Bootstrap starting.
I20260807 03:07:19.243407 1086 tablet_bootstrap.cc:492] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Bootstrap starting.
I20260807 03:07:19.244962 1085 tablet_bootstrap.cc:654] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:19.246240 1085 log.cc:824] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:19.246760 1087 tablet_bootstrap.cc:492] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Bootstrap starting.
I20260807 03:07:19.246981 1086 tablet_bootstrap.cc:654] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:19.248261 1086 log.cc:824] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:19.249692 1087 tablet_bootstrap.cc:654] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:19.250867 1087 log.cc:824] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:19.253079 1085 tablet_bootstrap.cc:492] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: No bootstrap required, opened a new log
I20260807 03:07:19.253075 1086 tablet_bootstrap.cc:492] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: No bootstrap required, opened a new log
I20260807 03:07:19.253243 1087 tablet_bootstrap.cc:492] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: No bootstrap required, opened a new log
I20260807 03:07:19.253357 1085 ts_tablet_manager.cc:1397] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260807 03:07:19.253373 1086 ts_tablet_manager.cc:1397] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 03:07:19.253489 1087 ts_tablet_manager.cc:1397] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260807 03:07:19.260950 1087 raft_consensus.cc:348] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.261301 1087 raft_consensus.cc:374] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:19.261413 1087 raft_consensus.cc:729] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1aaecfe640f40be9751f1280f6233fb, State: Initialized, Role: FOLLOWER
I20260807 03:07:19.261988 1087 consensus_queue.cc:260] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.263041 846 heartbeater.cc:503] Master 127.5.122.62:36835 was elected leader, sending a full tablet report...
I20260807 03:07:19.263535 1087 ts_tablet_manager.cc:1428] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260807 03:07:19.263661 1086 raft_consensus.cc:348] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.264074 1086 raft_consensus.cc:374] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:19.264186 1086 raft_consensus.cc:729] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f232551f4c4e32956df8750fa33ead, State: Initialized, Role: FOLLOWER
I20260807 03:07:19.264485 1085 raft_consensus.cc:348] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.264887 1085 raft_consensus.cc:374] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:19.264740 1086 consensus_queue.cc:260] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.264990 1085 raft_consensus.cc:729] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae830da71b6e4b12bb9ef6123daee264, State: Initialized, Role: FOLLOWER
I20260807 03:07:19.265527 1085 consensus_queue.cc:260] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.265733 1053 heartbeater.cc:503] Master 127.5.122.62:36835 was elected leader, sending a full tablet report...
I20260807 03:07:19.266477 1086 ts_tablet_manager.cc:1428] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260807 03:07:19.268043 1085 ts_tablet_manager.cc:1428] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
W20260807 03:07:19.398545 1054 tablet.cc:2369] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:07:19.401494 847 tablet.cc:2369] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:07:19.407960 648 tablet.cc:2369] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:07:19.432691 1092 raft_consensus.cc:482] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:19.432960 1092 raft_consensus.cc:504] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.434657 1092 leader_election.cc:288] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89f232551f4c4e32956df8750fa33ead (127.5.122.3:43855), ae830da71b6e4b12bb9ef6123daee264 (127.5.122.1:46197)
I20260807 03:07:19.441385 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.435083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:46033 (local address 127.5.122.1:46197)
0807 03:07:19.435287 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.435293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.436134 (+ 841us) server_negotiation.cc:407] Connection header received
0807 03:07:19.436273 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.436276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.436366 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.436458 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:19.437039 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.437723 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.439635 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.440417 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.440552 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.440650 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.441014 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.441138 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:07:19.441385 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.435136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:49559 (local address 127.5.122.3:43855)
0807 03:07:19.435372 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.435379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.435995 (+ 616us) server_negotiation.cc:407] Connection header received
0807 03:07:19.436172 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.436177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.436255 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.436331 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:19.436958 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.437721 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.439646 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.440398 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.440496 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.440601 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.441020 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.441145 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 03:07:19.441535 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.435080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:46197 (local address 127.5.122.2:46033)
0807 03:07:19.436015 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:19.436031 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:19.436161 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:19.436489 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:19.436493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:19.436526 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:19.436888 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.436900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.437860 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.437864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.439479 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.439491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.440657 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.440662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.440836 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.440886 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 03:07:19.440965 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 03:07:19.441093 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":90,"thread_start_us":493,"threads_started":1}
I20260807 03:07:19.441735 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.435014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:43855 (local address 127.5.122.2:49559)
0807 03:07:19.435879 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:19.435894 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:19.436031 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:19.436374 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:19.436378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:19.436403 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:19.436803 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.436814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.437863 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.437866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.439489 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:19.439502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.440622 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.440630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:19.440815 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.440889 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 03:07:19.440969 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 03:07:19.441098 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":92,"thread_start_us":482,"threads_started":1}
I20260807 03:07:19.442780 987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec82229b8424ee5a7295f701202132d" candidate_uuid: "a1aaecfe640f40be9751f1280f6233fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f232551f4c4e32956df8750fa33ead" is_pre_election: true
I20260807 03:07:19.442773 568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec82229b8424ee5a7295f701202132d" candidate_uuid: "a1aaecfe640f40be9751f1280f6233fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae830da71b6e4b12bb9ef6123daee264" is_pre_election: true
I20260807 03:07:19.443329 987 raft_consensus.cc:2454] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1aaecfe640f40be9751f1280f6233fb in term 0.
I20260807 03:07:19.443339 568 raft_consensus.cc:2454] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1aaecfe640f40be9751f1280f6233fb in term 0.
I20260807 03:07:19.444051 680 leader_election.cc:302] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1aaecfe640f40be9751f1280f6233fb, ae830da71b6e4b12bb9ef6123daee264; no voters:
I20260807 03:07:19.444423 1092 raft_consensus.cc:2781] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:07:19.444545 1092 raft_consensus.cc:482] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:07:19.444641 1092 raft_consensus.cc:3037] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:19.446834 1092 raft_consensus.cc:504] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.447469 1092 leader_election.cc:288] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [CANDIDATE]: Term 1 election: Requested vote from peers 89f232551f4c4e32956df8750fa33ead (127.5.122.3:43855), ae830da71b6e4b12bb9ef6123daee264 (127.5.122.1:46197)
I20260807 03:07:19.447767 987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec82229b8424ee5a7295f701202132d" candidate_uuid: "a1aaecfe640f40be9751f1280f6233fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f232551f4c4e32956df8750fa33ead"
I20260807 03:07:19.447952 987 raft_consensus.cc:3037] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:19.448096 568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec82229b8424ee5a7295f701202132d" candidate_uuid: "a1aaecfe640f40be9751f1280f6233fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae830da71b6e4b12bb9ef6123daee264"
I20260807 03:07:19.448276 568 raft_consensus.cc:3037] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:19.450073 987 raft_consensus.cc:2454] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1aaecfe640f40be9751f1280f6233fb in term 1.
I20260807 03:07:19.450510 678 leader_election.cc:302] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89f232551f4c4e32956df8750fa33ead, a1aaecfe640f40be9751f1280f6233fb; no voters:
I20260807 03:07:19.450618 568 raft_consensus.cc:2454] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1aaecfe640f40be9751f1280f6233fb in term 1.
I20260807 03:07:19.450762 1092 raft_consensus.cc:2781] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:19.451244 1092 raft_consensus.cc:686] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 1 LEADER]: Becoming Leader. State: Replica: a1aaecfe640f40be9751f1280f6233fb, State: Running, Role: LEADER
I20260807 03:07:19.451670 1092 consensus_queue.cc:237] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } }
I20260807 03:07:19.455960 353 catalog_manager.cc:5495] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb reported cstate change: term changed from 0 to 1, leader changed from <none> to a1aaecfe640f40be9751f1280f6233fb (127.5.122.2). New cstate: current_term: 1 leader_uuid: "a1aaecfe640f40be9751f1280f6233fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1aaecfe640f40be9751f1280f6233fb" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 34183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 } health_report { overall_health: UNKNOWN } } }
I20260807 03:07:19.523162 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.517778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.5.122.2:34183)
0807 03:07:19.517975 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.517982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.518057 (+ 75us) server_negotiation.cc:407] Connection header received
0807 03:07:19.518278 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.518282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.518402 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.518529 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:07:19.518918 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.519627 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.520978 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.521381 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.521491 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.522055 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:07:19.522578 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:07:19.522743 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.522900 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.522947 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:07:19.530622 568 raft_consensus.cc:1261] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 1 FOLLOWER]: Refusing update from remote peer a1aaecfe640f40be9751f1280f6233fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:19.530668 987 raft_consensus.cc:1261] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 1 FOLLOWER]: Refusing update from remote peer a1aaecfe640f40be9751f1280f6233fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:19.531535 1092 consensus_queue.cc:1035] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae830da71b6e4b12bb9ef6123daee264" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 46197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:19.531796 1100 consensus_queue.cc:1035] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f232551f4c4e32956df8750fa33ead" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 43855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:19.555768 355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:19.556032 355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:19.559303 355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0b775fb3f34436ba5abaa99c9b594bd: Sending DeleteTablet for 3 replicas of tablet 7ec82229b8424ee5a7295f701202132d
I20260807 03:07:19.560456 748 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec82229b8424ee5a7295f701202132d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:19 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:59470
I20260807 03:07:19.560580 535 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec82229b8424ee5a7295f701202132d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:19 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47390
I20260807 03:07:19.560642 953 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec82229b8424ee5a7295f701202132d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:19 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:49766
I20260807 03:07:19.564316 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 441
I20260807 03:07:19.571599 1120 tablet_replica.cc:330] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: stopping tablet replica
I20260807 03:07:19.572065 1120 raft_consensus.cc:2229] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:19.572603 1120 raft_consensus.cc:2258] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:19.576553 1120 ts_tablet_manager.cc:1905] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:19.581408 1122 tablet_replica.cc:330] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: stopping tablet replica
I20260807 03:07:19.581827 1122 raft_consensus.cc:2229] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:19.582264 1122 raft_consensus.cc:2258] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:19.584820 1117 tablet_replica.cc:330] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: stopping tablet replica
I20260807 03:07:19.587225 1117 raft_consensus.cc:2229] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:07:19.588171 1122 ts_tablet_manager.cc:1905] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:19.590876 1117 raft_consensus.cc:2258] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:19.593305 1117 ts_tablet_manager.cc:1905] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 03:07:19 dist-test-slave-fgb3 krb5kdc[32752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:07:19.596869 1120 ts_tablet_manager.cc:1918] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:19.597141 1120 log.cc:1196] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/7ec82229b8424ee5a7295f701202132d
I20260807 03:07:19.597350 1122 ts_tablet_manager.cc:1918] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:19.597543 1122 log.cc:1196] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/7ec82229b8424ee5a7295f701202132d
I20260807 03:07:19.597903 1120 ts_tablet_manager.cc:1939] T 7ec82229b8424ee5a7295f701202132d P ae830da71b6e4b12bb9ef6123daee264: Deleting consensus metadata
I20260807 03:07:19.598088 1122 ts_tablet_manager.cc:1939] T 7ec82229b8424ee5a7295f701202132d P 89f232551f4c4e32956df8750fa33ead: Deleting consensus metadata
I20260807 03:07:19.599932 332 catalog_manager.cc:4841] TS ae830da71b6e4b12bb9ef6123daee264 (127.5.122.1:46197): tablet 7ec82229b8424ee5a7295f701202132d (table test-table [id=9df24bda04b14e2bb974e716e066c9d8]) successfully deleted
I20260807 03:07:19.599952 330 catalog_manager.cc:4841] TS 89f232551f4c4e32956df8750fa33ead (127.5.122.3:43855): tablet 7ec82229b8424ee5a7295f701202132d (table test-table [id=9df24bda04b14e2bb974e716e066c9d8]) successfully deleted
I20260807 03:07:19.603576 1117 ts_tablet_manager.cc:1918] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:19.603730 1117 log.cc:1196] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/7ec82229b8424ee5a7295f701202132d
I20260807 03:07:19.604242 1117 ts_tablet_manager.cc:1939] T 7ec82229b8424ee5a7295f701202132d P a1aaecfe640f40be9751f1280f6233fb: Deleting consensus metadata
I20260807 03:07:19.605919 330 catalog_manager.cc:4841] TS a1aaecfe640f40be9751f1280f6233fb (127.5.122.2:34183): tablet 7ec82229b8424ee5a7295f701202132d (table test-table [id=9df24bda04b14e2bb974e716e066c9d8]) successfully deleted
I20260807 03:07:19.607764 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.565614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.5.122.1:46197)
0807 03:07:19.565783 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.565789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.578489 (+ 12700us) server_negotiation.cc:407] Connection header received
0807 03:07:19.578995 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.579002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.579091 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.579184 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:19.581172 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.582267 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.585047 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.585804 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.585921 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.599651 (+ 13730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:19.599670 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:19.599688 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:19.599751 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:19.603944 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.604991 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.605002 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.605006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.605187 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.605727 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.605733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.605751 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.606298 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:19.606664 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.607334 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.607505 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 03:07:19.608429 640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:19.743799 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 441
I20260807 03:07:19.767154 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 654
Aug 07 03:07:19 dist-test-slave-fgb3 krb5kdc[32752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:07:19.785228 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.768069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59484 (local address 127.5.122.2:34183)
0807 03:07:19.768235 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.768239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.768258 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:07:19.768461 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.768467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.768559 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.768667 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:19.770544 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.771364 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.773269 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.773692 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.773781 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.779225 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:19.779239 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:19.779254 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:19.779291 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:19.782221 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.783136 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.783144 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.783146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.783328 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.783754 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.783758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.783760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.784092 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:19.784378 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.784725 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.784886 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:07:19.785913 838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:19.897866 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 654
I20260807 03:07:19.921142 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 853
Aug 07 03:07:19 dist-test-slave-fgb3 krb5kdc[32752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:07:19.935365 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:19.921874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.5.122.3:43855)
0807 03:07:19.922031 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:19.922035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:19.922055 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:07:19.922217 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:19.922221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:19.922286 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:19.922415 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:19.923817 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.924518 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.925776 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:19.926168 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:19.926256 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:19.930480 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:19.930493 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:19.930509 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:19.930545 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:19.933093 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.933706 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.933712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.933714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.933843 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:19.934115 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:19.934117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:19.934119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:19.934494 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:19.934702 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:19.934953 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 03:07:19.935113 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:07:19.936002 1045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:20.024014 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 853
I20260807 03:07:20.045351 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 306
I20260807 03:07:20.046463 394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:20.153190 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 306
2026-08-07T03:07:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4416 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T03:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:20Z Disabled control of system clock
I20260807 03:07:20.236600 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42881
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:42881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 03:07:20.555871 1134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:20.556141 1134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:20.556268 1134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:20.563711 1134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:20.563810 1134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:20.563879 1134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:20.563957 1134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:20.564018 1134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:20.572938 1134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:42881
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:20.574143 1134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:20.575908 1134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:20.586903 1141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:20.586957 1144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:20.586961 1146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:20.587836 1134 server_base.cc:1034] running on GCE node
I20260807 03:07:20.588516 1134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:20.589774 1134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:20.590958 1134 hybrid_clock.cc:648] HybridClock initialized: now 1786072040590923 us; error 23 us; skew 500 ppm
I20260807 03:07:20.591248 1134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:20.593278 1134 webserver.cc:460] Webserver started at http://127.5.122.62:37139/ using document root <none> and password file <none>
I20260807 03:07:20.593876 1134 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:20.593966 1134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:20.594223 1134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:20.596956 1134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "ecb3775df62a4a10896cbdf773125d04"
format_stamp: "Formatted at 2026-08-07 03:07:20 on dist-test-slave-fgb3"
server_key: "661dcf1362faeebfb0bc47ca531021af"
server_key_iv: "9d2a951be953fd15a1a4cc843ef50799"
server_key_version: "encryptionkey@0"
I20260807 03:07:20.597707 1134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "ecb3775df62a4a10896cbdf773125d04"
format_stamp: "Formatted at 2026-08-07 03:07:20 on dist-test-slave-fgb3"
server_key: "661dcf1362faeebfb0bc47ca531021af"
server_key_iv: "9d2a951be953fd15a1a4cc843ef50799"
server_key_version: "encryptionkey@0"
I20260807 03:07:20.602415 1134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 03:07:20.605528 1154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:20.606755 1134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:07:20.606885 1134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "ecb3775df62a4a10896cbdf773125d04"
format_stamp: "Formatted at 2026-08-07 03:07:20 on dist-test-slave-fgb3"
server_key: "661dcf1362faeebfb0bc47ca531021af"
server_key_iv: "9d2a951be953fd15a1a4cc843ef50799"
server_key_version: "encryptionkey@0"
I20260807 03:07:20.607043 1134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:20.624776 1134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:20.625494 1134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:20.625722 1134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:20.642585 1134 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:42881
I20260807 03:07:20.642607 1229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:42881 every 8 connection(s)
I20260807 03:07:20.643864 1134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:20.647472 1230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:20.649832 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 1134
I20260807 03:07:20.650010 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:20.650568 5608 external_mini_cluster.cc:1428] Setting key 4c37e53948d0c4959a966de0793a0b85
I20260807 03:07:20.658883 1230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04: Bootstrap starting.
I20260807 03:07:20.661814 1230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:20.662812 1230 log.cc:824] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:20.663568 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:20.652488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.5.122.62:42881)
0807 03:07:20.653490 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:20.653501 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:20.653540 (+ 39us) server_negotiation.cc:407] Connection header received
0807 03:07:20.655209 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:20.655230 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:20.655647 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:20.656006 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:20.656550 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:20.657796 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:20.659229 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:20.659874 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:20.660070 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:20.661141 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:20.661148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:20.661161 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:20.661646 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:20.661738 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:20.661760 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:20.661875 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:20.662374 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 03:07:20.662625 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":365,"threads_started":1}
I20260807 03:07:20.666500 1230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04: No bootstrap required, opened a new log
I20260807 03:07:20.673524 1230 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb3775df62a4a10896cbdf773125d04" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42881 } }
I20260807 03:07:20.673854 1230 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:20.673954 1230 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb3775df62a4a10896cbdf773125d04, State: Initialized, Role: FOLLOWER
I20260807 03:07:20.674484 1230 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb3775df62a4a10896cbdf773125d04" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42881 } }
I20260807 03:07:20.674763 1230 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:20.674909 1230 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:20.675096 1230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:20.677166 1230 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb3775df62a4a10896cbdf773125d04" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42881 } }
I20260807 03:07:20.677615 1230 leader_election.cc:302] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecb3775df62a4a10896cbdf773125d04; no voters:
I20260807 03:07:20.678146 1230 leader_election.cc:288] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:20.678267 1239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:20.678965 1239 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [term 1 LEADER]: Becoming Leader. State: Replica: ecb3775df62a4a10896cbdf773125d04, State: Running, Role: LEADER
I20260807 03:07:20.679493 1239 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb3775df62a4a10896cbdf773125d04" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42881 } }
I20260807 03:07:20.680073 1230 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:20.682132 1242 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecb3775df62a4a10896cbdf773125d04. Latest consensus state: current_term: 1 leader_uuid: "ecb3775df62a4a10896cbdf773125d04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb3775df62a4a10896cbdf773125d04" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42881 } } }
I20260807 03:07:20.682099 1240 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecb3775df62a4a10896cbdf773125d04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb3775df62a4a10896cbdf773125d04" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42881 } } }
I20260807 03:07:20.682569 1240 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:20.682574 1242 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:20.684923 1250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:20.689406 1250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:20.696852 1250 catalog_manager.cc:1347] Generated new cluster ID: 3642cd46272f482eb8248ccb1c164381
I20260807 03:07:20.696964 1250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:20.713369 1250 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:20.715108 1250 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:20.725092 1250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04: Generated new TSK 0
I20260807 03:07:20.726064 1250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:20.730554 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42881
--builtin_ntp_servers=127.5.122.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 03:07:21.053768 1273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:21.054037 1273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:21.054159 1273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:21.061205 1273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:21.061313 1273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:21.061467 1273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:07:21.070245 1273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:21.071492 1273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:21.073194 1273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:21.085340 1282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:21.085309 1283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:21.085378 1287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:21.086560 1273 server_base.cc:1034] running on GCE node
I20260807 03:07:21.087082 1273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:21.087898 1273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:21.089095 1273 hybrid_clock.cc:648] HybridClock initialized: now 1786072041089057 us; error 26 us; skew 500 ppm
I20260807 03:07:21.089377 1273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:21.091464 1273 webserver.cc:460] Webserver started at http://127.5.122.1:33665/ using document root <none> and password file <none>
I20260807 03:07:21.092080 1273 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:21.092177 1273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:21.092478 1273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:21.095155 1273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "264356cbebd14d99b79127d8b159922b"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "c8534f78ae67a4bbee4987824adb642f"
server_key_iv: "c0a03d0af34737c5cd4633b96f37afce"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.095882 1273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "264356cbebd14d99b79127d8b159922b"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "c8534f78ae67a4bbee4987824adb642f"
server_key_iv: "c0a03d0af34737c5cd4633b96f37afce"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.100531 1273 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:07:21.103643 1295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:21.104779 1273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 03:07:21.104923 1273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "264356cbebd14d99b79127d8b159922b"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "c8534f78ae67a4bbee4987824adb642f"
server_key_iv: "c0a03d0af34737c5cd4633b96f37afce"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.105108 1273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:21.131719 1273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:21.132481 1273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:21.132720 1273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:21.133599 1273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:21.134976 1273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:21.135068 1273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:21.135174 1273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:21.135242 1273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:21.164310 1273 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:34313
I20260807 03:07:21.164371 1461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:34313 every 8 connection(s)
I20260807 03:07:21.165464 1273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:07:21.170931 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 1273
I20260807 03:07:21.171082 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:07:21.171497 5608 external_mini_cluster.cc:1428] Setting key e2796552844d8e91c463ada860f14e05
I20260807 03:07:21.174381 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42881
--builtin_ntp_servers=127.5.122.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 03:07:21.177549 1466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:21.167723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:42611 (local address 127.5.122.62:42881)
0807 03:07:21.168144 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:21.168148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:21.169146 (+ 998us) server_negotiation.cc:407] Connection header received
0807 03:07:21.170947 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:21.170953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:21.171044 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:21.171143 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:21.172252 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.173100 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.174982 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.175569 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.175695 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:21.176806 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:21.176811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:21.176815 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:21.176834 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:21.176892 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:21.176907 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:21.176986 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:21.177152 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 03:07:21.177243 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":226,"threads_started":1}
I20260807 03:07:21.178436 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:21.168157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42881 (local address 127.5.122.1:42611)
0807 03:07:21.168991 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:21.169031 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:21.170693 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:21.171383 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:21.171390 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:21.171479 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:21.172082 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:21.172103 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.173299 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.173304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:21.174847 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:21.174859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.175749 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.175753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:21.175940 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:21.176530 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:21.176533 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:21.176572 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:21.176639 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:21.176991 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:21.176997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:21.177003 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 03:07:21.177127 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 03:07:21.177374 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":263,"threads_started":1}
I20260807 03:07:21.179826 1462 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42881
I20260807 03:07:21.180102 1462 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:21.180904 1462 heartbeater.cc:511] Master 127.5.122.62:42881 requested a full tablet report, sending...
I20260807 03:07:21.183049 1178 ts_manager.cc:194] Registered new tserver with Master: 264356cbebd14d99b79127d8b159922b (127.5.122.1:34313)
I20260807 03:07:21.184993 1178 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.1:42611
W20260807 03:07:21.503037 1469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:21.503314 1469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:21.503439 1469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:21.510725 1469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:21.510860 1469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:21.511029 1469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:07:21.520153 1469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:21.521418 1469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:21.523167 1469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:21.535538 1477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:21.535473 1479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:21.535517 1486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:21.536154 1469 server_base.cc:1034] running on GCE node
I20260807 03:07:21.536728 1469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:21.537583 1469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:21.538777 1469 hybrid_clock.cc:648] HybridClock initialized: now 1786072041538722 us; error 50 us; skew 500 ppm
I20260807 03:07:21.539068 1469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:21.541136 1469 webserver.cc:460] Webserver started at http://127.5.122.2:35985/ using document root <none> and password file <none>
I20260807 03:07:21.541778 1469 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:21.541878 1469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:21.542171 1469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:21.544906 1469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "d3e369ca874b440c90cd6c2606388b2a"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "ae346d5655515a3bbedf8d26fa147e28"
server_key_iv: "c97bfd2dd6554a870814fd694edf99a0"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.545650 1469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "d3e369ca874b440c90cd6c2606388b2a"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "ae346d5655515a3bbedf8d26fa147e28"
server_key_iv: "c97bfd2dd6554a870814fd694edf99a0"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.550631 1469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260807 03:07:21.553869 1494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:21.555138 1469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:07:21.555284 1469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "d3e369ca874b440c90cd6c2606388b2a"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "ae346d5655515a3bbedf8d26fa147e28"
server_key_iv: "c97bfd2dd6554a870814fd694edf99a0"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.555481 1469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:21.568079 1469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:21.568775 1469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:21.569020 1469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:21.569896 1469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:21.571254 1469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:21.571346 1469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:21.571456 1469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:21.571527 1469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:21.600093 1469 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:33683
I20260807 03:07:21.600139 1661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:33683 every 8 connection(s)
I20260807 03:07:21.601295 1469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:07:21.604868 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 1469
I20260807 03:07:21.605050 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:07:21.605621 5608 external_mini_cluster.cc:1428] Setting key 841e477c7f7b701194f5a70cd03e5402
I20260807 03:07:21.608566 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42881
--builtin_ntp_servers=127.5.122.20:39167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 03:07:21.613804 1466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:21.603657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:59747 (local address 127.5.122.62:42881)
0807 03:07:21.603842 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:21.603847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:21.605402 (+ 1555us) server_negotiation.cc:407] Connection header received
0807 03:07:21.607545 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:21.607550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:21.607636 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:21.607734 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:21.608832 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.609601 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.611278 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.611731 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.611841 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:21.613049 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:21.613053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:21.613057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:21.613074 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:21.613104 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:21.613116 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:21.613187 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:21.613387 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 03:07:21.613485 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:07:21.614768 1664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:21.604162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42881 (local address 127.5.122.2:59747)
0807 03:07:21.605224 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:21.605279 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:21.607285 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:21.607962 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:21.607970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:21.608050 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:21.608651 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:21.608675 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.609758 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.609762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:21.611140 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:21.611152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:21.611941 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:21.611949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:21.612160 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:21.612769 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:21.612772 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:21.612808 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:21.612877 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:21.613199 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:21.613210 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:21.613218 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 03:07:21.613362 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 03:07:21.613732 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":318,"threads_started":1}
I20260807 03:07:21.616089 1662 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42881
I20260807 03:07:21.616338 1662 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:21.616998 1662 heartbeater.cc:511] Master 127.5.122.62:42881 requested a full tablet report, sending...
I20260807 03:07:21.618382 1178 ts_manager.cc:194] Registered new tserver with Master: d3e369ca874b440c90cd6c2606388b2a (127.5.122.2:33683)
I20260807 03:07:21.619376 1178 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.2:59747
W20260807 03:07:21.929021 1667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:21.929294 1667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:21.929418 1667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:21.936305 1667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:21.936401 1667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:21.936539 1667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:07:21.945286 1667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:39167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:42881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:21.946497 1667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:21.948169 1667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:21.960201 1674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:21.960125 1681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:21.960157 1679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:21.961310 1667 server_base.cc:1034] running on GCE node
I20260807 03:07:21.961859 1667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:21.962783 1667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:21.963977 1667 hybrid_clock.cc:648] HybridClock initialized: now 1786072041963939 us; error 29 us; skew 500 ppm
I20260807 03:07:21.964239 1667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:21.966261 1667 webserver.cc:460] Webserver started at http://127.5.122.3:37863/ using document root <none> and password file <none>
I20260807 03:07:21.966923 1667 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:21.967013 1667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:21.967273 1667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:21.969923 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "eba3493217ca4f08976ee63f63177438"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "d2e07887b1dcaf797b6d4c5ca04c790f"
server_key_iv: "b069b9d41e50ef9d8f1115ec0ad14537"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.970672 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "eba3493217ca4f08976ee63f63177438"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "d2e07887b1dcaf797b6d4c5ca04c790f"
server_key_iv: "b069b9d41e50ef9d8f1115ec0ad14537"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.975306 1667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 03:07:21.978577 1691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:21.979817 1667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:07:21.979959 1667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "eba3493217ca4f08976ee63f63177438"
format_stamp: "Formatted at 2026-08-07 03:07:21 on dist-test-slave-fgb3"
server_key: "d2e07887b1dcaf797b6d4c5ca04c790f"
server_key_iv: "b069b9d41e50ef9d8f1115ec0ad14537"
server_key_version: "encryptionkey@0"
I20260807 03:07:21.980150 1667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:22.008653 1667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:22.009532 1667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:22.009857 1667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:22.011243 1667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:22.013312 1667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:22.013399 1667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:22.013481 1667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:22.013540 1667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:22.043337 1667 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:40119
I20260807 03:07:22.043411 1846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:40119 every 8 connection(s)
I20260807 03:07:22.044615 1667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:07:22.047685 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 1667
I20260807 03:07:22.047880 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:07:22.048451 5608 external_mini_cluster.cc:1428] Setting key f8ca52ad9bf68553514766768a665325
I20260807 03:07:22.056317 1466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.047090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:59377 (local address 127.5.122.62:42881)
0807 03:07:22.047308 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.047313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.048764 (+ 1451us) server_negotiation.cc:407] Connection header received
0807 03:07:22.050569 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.050575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.050656 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.050754 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:22.051811 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.052530 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.054038 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.054477 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.054574 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.055712 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:22.055717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:22.055720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:22.055737 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:22.055768 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:22.055779 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:22.055845 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.056010 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.056083 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 03:07:22.057345 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.047637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:42881 (local address 127.5.122.3:59377)
0807 03:07:22.048610 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.048655 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:22.050255 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:22.050988 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:22.051000 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:22.051100 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:22.051654 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.051670 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.052659 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.052663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.053913 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.053926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.054647 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.054652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.054852 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.055416 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:22.055418 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:22.055455 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:22.055531 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:22.055852 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:22.055859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:22.055864 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:07:22.055995 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 03:07:22.056273 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":317,"threads_started":1}
I20260807 03:07:22.058684 1848 heartbeater.cc:347] Connected to a master server at 127.5.122.62:42881
I20260807 03:07:22.058931 1848 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:22.059662 1848 heartbeater.cc:511] Master 127.5.122.62:42881 requested a full tablet report, sending...
I20260807 03:07:22.060914 1178 ts_manager.cc:194] Registered new tserver with Master: eba3493217ca4f08976ee63f63177438 (127.5.122.3:40119)
I20260807 03:07:22.061795 1178 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.3:59377
I20260807 03:07:22.063555 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:07:22.073109 1466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.067749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.5.122.62:42881)
0807 03:07:22.067935 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.067943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.068344 (+ 401us) server_negotiation.cc:407] Connection header received
0807 03:07:22.068441 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.068445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.068519 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.068600 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:22.069067 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.069768 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.071078 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.071447 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.071537 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.072497 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:22.072502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:22.072505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:22.072522 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:22.072552 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:22.072564 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:22.072670 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.072765 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.072806 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:07:22.076648 1177 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:07:22.083595 1177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:07:22.105171 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.095567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.5.122.3:40119)
0807 03:07:22.096546 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.096555 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.097213 (+ 658us) server_negotiation.cc:407] Connection header received
0807 03:07:22.097321 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.097329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.097631 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.097814 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:22.098827 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.100342 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.102683 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.103786 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.103911 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.104194 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.104679 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.104834 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":618,"threads_started":1}
I20260807 03:07:22.105158 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.094971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:40119 (local address 127.0.0.1:34330)
0807 03:07:22.096970 (+ 1999us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.096989 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:22.097097 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:22.098138 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:22.098145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:22.098171 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:22.098618 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.098632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.100519 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.100524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.102513 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.102525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.104313 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.104319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.104470 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.104501 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 03:07:22.104605 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 03:07:22.104774 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1852,"mutex_wait_us":1,"thread_start_us":1726,"threads_started":1}
I20260807 03:07:22.105921 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.094770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:33683 (local address 127.0.0.1:51738)
0807 03:07:22.095698 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.095741 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:22.095913 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:22.099190 (+ 3277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:22.099194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:22.099218 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:22.099638 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.099651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.101310 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.101315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.103448 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.103463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.105231 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.105238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.105387 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.105420 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:07:22.105486 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 03:07:22.105608 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":496,"threads_started":1}
I20260807 03:07:22.106590 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.095639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57014)
0807 03:07:22.098049 (+ 2410us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.098065 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:22.098159 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:22.099464 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:22.099469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:22.099497 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:22.099931 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.099952 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.101780 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.101785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.103898 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.103912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.105912 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.105919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.106080 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.106112 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:07:22.106190 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 03:07:22.106296 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2282,"thread_start_us":2132,"threads_started":1}
I20260807 03:07:22.106674 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.095098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.5.122.2:33683)
0807 03:07:22.098639 (+ 3541us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.098646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.098674 (+ 28us) server_negotiation.cc:407] Connection header received
0807 03:07:22.098769 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.098777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.098998 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.099189 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:22.099838 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.101192 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.103664 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.105007 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.105805 (+ 798us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.106053 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.106122 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.106265 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3383,"thread_start_us":3254,"threads_started":1}
I20260807 03:07:22.106947 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.096204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.5.122.1:34313)
0807 03:07:22.098915 (+ 2711us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.098920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.098942 (+ 22us) server_negotiation.cc:407] Connection header received
0807 03:07:22.099018 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.099023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.099177 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.099369 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:22.100141 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.101638 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.104109 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.105453 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.105567 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.105780 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.106452 (+ 672us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.106609 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2583,"thread_start_us":2474,"threads_started":1}
I20260807 03:07:22.109483 1752 tablet_service.cc:1467] Processing CreateTablet for tablet 4b9796b23a3b45e386c6a78d6df0c654 (DEFAULT_TABLE table=test-table [id=3056bf2ebf6e4321b2e584848be9d8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:22.110033 1367 tablet_service.cc:1467] Processing CreateTablet for tablet 4b9796b23a3b45e386c6a78d6df0c654 (DEFAULT_TABLE table=test-table [id=3056bf2ebf6e4321b2e584848be9d8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:22.110157 1569 tablet_service.cc:1467] Processing CreateTablet for tablet 4b9796b23a3b45e386c6a78d6df0c654 (DEFAULT_TABLE table=test-table [id=3056bf2ebf6e4321b2e584848be9d8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:22.111096 1752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b9796b23a3b45e386c6a78d6df0c654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:22.111281 1367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b9796b23a3b45e386c6a78d6df0c654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:22.111450 1569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b9796b23a3b45e386c6a78d6df0c654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:22.122071 1872 tablet_bootstrap.cc:492] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: Bootstrap starting.
I20260807 03:07:22.124830 1873 tablet_bootstrap.cc:492] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Bootstrap starting.
I20260807 03:07:22.126195 1874 tablet_bootstrap.cc:492] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Bootstrap starting.
I20260807 03:07:22.125337 1872 tablet_bootstrap.cc:654] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:22.127758 1872 log.cc:824] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:22.128134 1873 tablet_bootstrap.cc:654] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:22.128907 1874 tablet_bootstrap.cc:654] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:22.129365 1873 log.cc:824] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:22.130019 1874 log.cc:824] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:22.131592 1872 tablet_bootstrap.cc:492] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: No bootstrap required, opened a new log
I20260807 03:07:22.131863 1872 ts_tablet_manager.cc:1397] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 03:07:22.132110 1873 tablet_bootstrap.cc:492] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: No bootstrap required, opened a new log
I20260807 03:07:22.132347 1873 ts_tablet_manager.cc:1397] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 03:07:22.133260 1874 tablet_bootstrap.cc:492] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: No bootstrap required, opened a new log
I20260807 03:07:22.133508 1874 ts_tablet_manager.cc:1397] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 03:07:22.139308 1873 raft_consensus.cc:348] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.139655 1873 raft_consensus.cc:374] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:22.139750 1873 raft_consensus.cc:729] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e369ca874b440c90cd6c2606388b2a, State: Initialized, Role: FOLLOWER
I20260807 03:07:22.140261 1873 consensus_queue.cc:260] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.141264 1662 heartbeater.cc:503] Master 127.5.122.62:42881 was elected leader, sending a full tablet report...
I20260807 03:07:22.141749 1873 ts_tablet_manager.cc:1428] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260807 03:07:22.141891 1872 raft_consensus.cc:348] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.142295 1872 raft_consensus.cc:374] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:22.142419 1872 raft_consensus.cc:729] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264356cbebd14d99b79127d8b159922b, State: Initialized, Role: FOLLOWER
I20260807 03:07:22.142949 1872 consensus_queue.cc:260] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.143891 1462 heartbeater.cc:503] Master 127.5.122.62:42881 was elected leader, sending a full tablet report...
I20260807 03:07:22.143976 1874 raft_consensus.cc:348] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.144371 1874 raft_consensus.cc:374] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:22.144459 1874 raft_consensus.cc:729] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba3493217ca4f08976ee63f63177438, State: Initialized, Role: FOLLOWER
I20260807 03:07:22.144523 1872 ts_tablet_manager.cc:1428] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 03:07:22.145370 1874 consensus_queue.cc:260] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.146401 1848 heartbeater.cc:503] Master 127.5.122.62:42881 was elected leader, sending a full tablet report...
I20260807 03:07:22.147102 1874 ts_tablet_manager.cc:1428] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260807 03:07:22.167567 1465 tablet.cc:2369] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:07:22.193639 1884 raft_consensus.cc:482] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:22.193959 1884 raft_consensus.cc:504] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.195664 1884 leader_election.cc:288] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3e369ca874b440c90cd6c2606388b2a (127.5.122.2:33683), eba3493217ca4f08976ee63f63177438 (127.5.122.3:40119)
I20260807 03:07:22.202831 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.196345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:36187 (local address 127.5.122.3:40119)
0807 03:07:22.196550 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.196556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.197055 (+ 499us) server_negotiation.cc:407] Connection header received
0807 03:07:22.197314 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.197319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.197400 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.197463 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:22.198285 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.199153 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.200855 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.201896 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.202019 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.202114 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.202461 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.202589 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 03:07:22.202931 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.196018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:40119 (local address 127.5.122.1:36187)
0807 03:07:22.196968 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.196984 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:22.197155 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:22.197660 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:22.197665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:22.197690 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:22.198090 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.198103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.199271 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.199277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.200676 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.200688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.202089 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.202096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.202273 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.202326 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 03:07:22.202418 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 03:07:22.202551 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":26,"thread_start_us":267,"threads_started":1}
I20260807 03:07:22.203402 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.196494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:42419 (local address 127.5.122.2:33683)
0807 03:07:22.196669 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.196673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.196887 (+ 214us) server_negotiation.cc:407] Connection header received
0807 03:07:22.196952 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.196956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.197013 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.197099 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:22.197713 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.198652 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.201250 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.202372 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.202489 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.202583 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.203046 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.203171 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:07:22.203961 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.196016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:33683 (local address 127.5.122.1:42419)
0807 03:07:22.196651 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.196670 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:22.196848 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:22.197256 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:22.197259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:22.197285 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:22.197579 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.197590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.198819 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.198825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.201036 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:22.201051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.202705 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.202711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:22.202865 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.202900 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 03:07:22.203567 (+ 667us) client_negotiation.cc:240] Negotiation successful
0807 03:07:22.203693 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":1,"thread_start_us":655,"threads_started":1}
I20260807 03:07:22.204035 1784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9796b23a3b45e386c6a78d6df0c654" candidate_uuid: "264356cbebd14d99b79127d8b159922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba3493217ca4f08976ee63f63177438" is_pre_election: true
I20260807 03:07:22.204559 1784 raft_consensus.cc:2454] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 264356cbebd14d99b79127d8b159922b in term 0.
I20260807 03:07:22.204766 1604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9796b23a3b45e386c6a78d6df0c654" candidate_uuid: "264356cbebd14d99b79127d8b159922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e369ca874b440c90cd6c2606388b2a" is_pre_election: true
I20260807 03:07:22.205210 1297 leader_election.cc:302] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264356cbebd14d99b79127d8b159922b, eba3493217ca4f08976ee63f63177438; no voters:
I20260807 03:07:22.205295 1604 raft_consensus.cc:2454] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 264356cbebd14d99b79127d8b159922b in term 0.
I20260807 03:07:22.205595 1884 raft_consensus.cc:2781] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:07:22.205729 1884 raft_consensus.cc:482] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:07:22.205826 1884 raft_consensus.cc:3037] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:22.208194 1884 raft_consensus.cc:504] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.208845 1884 leader_election.cc:288] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [CANDIDATE]: Term 1 election: Requested vote from peers d3e369ca874b440c90cd6c2606388b2a (127.5.122.2:33683), eba3493217ca4f08976ee63f63177438 (127.5.122.3:40119)
I20260807 03:07:22.209410 1784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9796b23a3b45e386c6a78d6df0c654" candidate_uuid: "264356cbebd14d99b79127d8b159922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba3493217ca4f08976ee63f63177438"
I20260807 03:07:22.209613 1784 raft_consensus.cc:3037] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:22.209614 1604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9796b23a3b45e386c6a78d6df0c654" candidate_uuid: "264356cbebd14d99b79127d8b159922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e369ca874b440c90cd6c2606388b2a"
I20260807 03:07:22.209826 1604 raft_consensus.cc:3037] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:22.212795 1784 raft_consensus.cc:2454] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 264356cbebd14d99b79127d8b159922b in term 1.
I20260807 03:07:22.212881 1604 raft_consensus.cc:2454] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 264356cbebd14d99b79127d8b159922b in term 1.
I20260807 03:07:22.213251 1297 leader_election.cc:302] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264356cbebd14d99b79127d8b159922b, eba3493217ca4f08976ee63f63177438; no voters:
I20260807 03:07:22.213524 1884 raft_consensus.cc:2781] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:22.214017 1884 raft_consensus.cc:686] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 1 LEADER]: Becoming Leader. State: Replica: 264356cbebd14d99b79127d8b159922b, State: Running, Role: LEADER
I20260807 03:07:22.214605 1884 consensus_queue.cc:237] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } }
I20260807 03:07:22.219367 1178 catalog_manager.cc:5495] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b reported cstate change: term changed from 0 to 1, leader changed from <none> to 264356cbebd14d99b79127d8b159922b (127.5.122.1). New cstate: current_term: 1 leader_uuid: "264356cbebd14d99b79127d8b159922b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "264356cbebd14d99b79127d8b159922b" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 34313 } health_report { overall_health: HEALTHY } } }
I20260807 03:07:22.259591 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.254489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.5.122.1:34313)
0807 03:07:22.254661 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.254665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.254683 (+ 18us) server_negotiation.cc:407] Connection header received
0807 03:07:22.254845 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.254848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.254947 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.255070 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:07:22.255417 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.256193 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.257576 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.257979 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.258088 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.258589 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:07:22.259105 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:07:22.259251 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.259346 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.259383 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 03:07:22.267603 1784 raft_consensus.cc:1261] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 1 FOLLOWER]: Refusing update from remote peer 264356cbebd14d99b79127d8b159922b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:22.267603 1604 raft_consensus.cc:1261] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 1 FOLLOWER]: Refusing update from remote peer 264356cbebd14d99b79127d8b159922b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:22.268573 1890 consensus_queue.cc:1035] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba3493217ca4f08976ee63f63177438" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 40119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:22.268901 1884 consensus_queue.cc:1035] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e369ca874b440c90cd6c2606388b2a" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:22.279089 1896 mvcc.cc:205] Tried to move back new op lower bound from 7315751085118488576 to 7315751084915572736. Current Snapshot: MvccSnapshot[applied={T|T < 7315751085118488576}]
I20260807 03:07:22.281227 1897 mvcc.cc:205] Tried to move back new op lower bound from 7315751085118488576 to 7315751084915572736. Current Snapshot: MvccSnapshot[applied={T|T < 7315751085118488576}]
I20260807 03:07:22.294130 1177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:22.294445 1177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:22.297859 1177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecb3775df62a4a10896cbdf773125d04: Sending DeleteTablet for 3 replicas of tablet 4b9796b23a3b45e386c6a78d6df0c654
I20260807 03:07:22.298964 1569 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b9796b23a3b45e386c6a78d6df0c654 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:22 UTC) from {username='slave'} at 127.0.0.1:51738
I20260807 03:07:22.299060 1752 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b9796b23a3b45e386c6a78d6df0c654 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:22 UTC) from {username='slave'} at 127.0.0.1:34330
I20260807 03:07:22.299238 1367 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b9796b23a3b45e386c6a78d6df0c654 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:22 UTC) from {username='slave'} at 127.0.0.1:57014
I20260807 03:07:22.301102 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 1273
I20260807 03:07:22.311329 1909 tablet_replica.cc:330] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: stopping tablet replica
I20260807 03:07:22.312387 1909 raft_consensus.cc:2229] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:22.312911 1909 raft_consensus.cc:2258] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:22.316228 1909 ts_tablet_manager.cc:1905] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:22.319736 1910 tablet_replica.cc:330] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: stopping tablet replica
I20260807 03:07:22.320293 1910 raft_consensus.cc:2229] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:22.320873 1910 raft_consensus.cc:2258] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:22.323031 1910 ts_tablet_manager.cc:1905] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:22.323837 1911 tablet_replica.cc:330] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: stopping tablet replica
I20260807 03:07:22.324167 1911 raft_consensus.cc:2229] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:07:22.324688 1911 raft_consensus.cc:2258] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:22.327870 1911 ts_tablet_manager.cc:1905] T 4b9796b23a3b45e386c6a78d6df0c654 P 264356cbebd14d99b79127d8b159922b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:22.330369 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.302452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57032 (local address 127.5.122.1:34313)
0807 03:07:22.302627 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.302634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.318839 (+ 16205us) server_negotiation.cc:407] Connection header received
0807 03:07:22.319122 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.319129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.319234 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.319354 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:22.320050 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.321368 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.325038 (+ 3670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.325523 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.325600 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.328559 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:22.328570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:22.328587 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:22.329399 (+ 812us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:22.329475 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:22.329501 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:22.329578 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.329837 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.329946 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:07:22.331524 1909 ts_tablet_manager.cc:1918] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:22.331705 1457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:22.331768 1909 log.cc:1196] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/4b9796b23a3b45e386c6a78d6df0c654
I20260807 03:07:22.332288 1909 ts_tablet_manager.cc:1939] T 4b9796b23a3b45e386c6a78d6df0c654 P d3e369ca874b440c90cd6c2606388b2a: Deleting consensus metadata
I20260807 03:07:22.332721 1910 ts_tablet_manager.cc:1918] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:22.333014 1910 log.cc:1196] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/4b9796b23a3b45e386c6a78d6df0c654
I20260807 03:07:22.333869 1910 ts_tablet_manager.cc:1939] T 4b9796b23a3b45e386c6a78d6df0c654 P eba3493217ca4f08976ee63f63177438: Deleting consensus metadata
I20260807 03:07:22.334518 1157 catalog_manager.cc:4841] TS d3e369ca874b440c90cd6c2606388b2a (127.5.122.2:33683): tablet 4b9796b23a3b45e386c6a78d6df0c654 (table test-table [id=3056bf2ebf6e4321b2e584848be9d8f1]) successfully deleted
I20260807 03:07:22.335848 1155 catalog_manager.cc:4841] TS eba3493217ca4f08976ee63f63177438 (127.5.122.3:40119): tablet 4b9796b23a3b45e386c6a78d6df0c654 (table test-table [id=3056bf2ebf6e4321b2e584848be9d8f1]) successfully deleted
I20260807 03:07:22.427857 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 1273
W20260807 03:07:22.450666 1159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.1:34313 (error 108)
I20260807 03:07:22.450986 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 1469
I20260807 03:07:22.452638 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.452163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57038)
0807 03:07:22.452350 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.452461 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 03:07:22.452876 1159 catalog_manager.cc:4584] TS 264356cbebd14d99b79127d8b159922b (127.5.122.1:34313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b9796b23a3b45e386c6a78d6df0c654: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
I20260807 03:07:22.458016 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.451880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51750 (local address 127.5.122.2:33683)
0807 03:07:22.452064 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.452070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.452091 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:07:22.452344 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.452349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.452449 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.452527 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:22.453107 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.453880 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.455202 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.455589 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.455677 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.456747 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:22.456756 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:22.456769 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:22.457321 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:22.457366 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:22.457385 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:22.457474 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.457649 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.457747 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 03:07:22.458818 1656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:22.506585 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.506002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57044)
0807 03:07:22.506245 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.506402 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 03:07:22.507721 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.507340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57058)
0807 03:07:22.507501 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.507577 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 03:07:22.545344 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 1469
I20260807 03:07:22.565704 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 1667
I20260807 03:07:22.568293 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.567802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57060)
0807 03:07:22.568020 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.568109 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 03:07:22.569283 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.568960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57072)
0807 03:07:22.569105 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.569170 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 03:07:22.572715 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.566487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.5.122.3:40119)
0807 03:07:22.566647 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:22.566652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:22.566724 (+ 72us) server_negotiation.cc:407] Connection header received
0807 03:07:22.566904 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:22.566909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:22.567015 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:22.567101 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:22.567552 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.568435 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.569992 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:22.570434 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:22.570528 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:22.571526 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:22.571535 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:22.571550 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:22.572100 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:22.572148 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:22.572168 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:22.572262 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:22.572409 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 03:07:22.572500 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 03:07:22.573386 1842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:22.650043 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.649447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57076)
0807 03:07:22.649715 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.649836 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 03:07:22.651432 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:22.650928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:34313 (local address 127.0.0.1:57088)
0807 03:07:22.651169 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:22.651287 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:34313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 03:07:22.660651 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 1667
I20260807 03:07:22.680132 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 1134
I20260807 03:07:22.681265 1224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:22.788149 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 1134
2026-08-07T03:07:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2636 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:22 dist-test-slave-fgb3 krb5kdc[1925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:22 dist-test-slave-fgb3 krb5kdc[1925](info): set up 2 sockets
Aug 07 03:07:22 dist-test-slave-fgb3 krb5kdc[1925](info): commencing operation
krb5kdc: starting...
W20260807 03:07:23.993294 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:24 dist-test-slave-fgb3 krb5kdc[1925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:07:24.273548 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40241
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:24.590951 1939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:24.591221 1939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:24.591341 1939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:24.598425 1939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:24.598537 1939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:24.598608 1939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:24.598685 1939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:24.598745 1939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:24.608420 1939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:24.609668 1939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:24.610976 1939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:24.621968 1946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:24.622030 1947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:24.621937 1949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:24.623345 1939 server_base.cc:1034] running on GCE node
I20260807 03:07:24.624078 1939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:24.625361 1939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:24.626550 1939 hybrid_clock.cc:648] HybridClock initialized: now 1786072044626512 us; error 28 us; skew 500 ppm
I20260807 03:07:24.626843 1939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:24 dist-test-slave-fgb3 krb5kdc[1925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072044, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:24.632342 1939 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:07:24.633811 1939 webserver.cc:460] Webserver started at http://127.5.122.62:36027/ using document root <none> and password file <none>
I20260807 03:07:24.634477 1939 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:24.634582 1939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:24.634845 1939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:24.637739 1939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "91bd4dbbe4784df28a0f6e1e4b29501b"
format_stamp: "Formatted at 2026-08-07 03:07:24 on dist-test-slave-fgb3"
server_key: "a7491267f0115b30fb53e80c903c5361"
server_key_iv: "57609c06d03d9ae7f580e5434b1e4cfd"
server_key_version: "encryptionkey@0"
I20260807 03:07:24.638525 1939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "91bd4dbbe4784df28a0f6e1e4b29501b"
format_stamp: "Formatted at 2026-08-07 03:07:24 on dist-test-slave-fgb3"
server_key: "a7491267f0115b30fb53e80c903c5361"
server_key_iv: "57609c06d03d9ae7f580e5434b1e4cfd"
server_key_version: "encryptionkey@0"
I20260807 03:07:24.643417 1939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:07:24.646643 1957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:24.647854 1939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:07:24.648008 1939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "91bd4dbbe4784df28a0f6e1e4b29501b"
format_stamp: "Formatted at 2026-08-07 03:07:24 on dist-test-slave-fgb3"
server_key: "a7491267f0115b30fb53e80c903c5361"
server_key_iv: "57609c06d03d9ae7f580e5434b1e4cfd"
server_key_version: "encryptionkey@0"
I20260807 03:07:24.648180 1939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:24.668427 1939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:24.672277 1939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:24.672544 1939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:24.689324 1939 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:40241
I20260807 03:07:24.689347 2032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:40241 every 8 connection(s)
I20260807 03:07:24.690861 1939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:24.694720 2033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:24.698994 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 1939
I20260807 03:07:24.699170 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:24.699601 5608 external_mini_cluster.cc:1428] Setting key 8d63384dda3b711ad179c226ba16794b
I20260807 03:07:24.706977 2033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b: Bootstrap starting.
I20260807 03:07:24.710626 2033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:24.711854 2033 log.cc:824] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:24.715292 2033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b: No bootstrap required, opened a new log
Aug 07 03:07:24 dist-test-slave-fgb3 krb5kdc[1925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:24.725226 2033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40241 } }
I20260807 03:07:24.725724 2033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:24.725838 2033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91bd4dbbe4784df28a0f6e1e4b29501b, State: Initialized, Role: FOLLOWER
I20260807 03:07:24.725955 2037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:24.701482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.5.122.62:40241)
0807 03:07:24.702280 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:24.702295 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:24.702378 (+ 83us) server_negotiation.cc:407] Connection header received
0807 03:07:24.705216 (+ 2838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:24.705246 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:24.705735 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:24.706193 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:24.707640 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:24.709607 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:24.711058 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:24.712102 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:24.712276 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:24.717129 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:24.717140 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:24.717158 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:24.717212 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:24.721187 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:24.721969 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:24.721978 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:24.721987 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:24.722202 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:24.722616 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:24.722621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:24.722627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:24.723372 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:24.723666 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:24.724083 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 03:07:24.724542 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":328,"threads_started":1}
I20260807 03:07:24.726539 2033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40241 } }
I20260807 03:07:24.726792 2033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:24.726925 2033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:24.727093 2033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:24.729611 2033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40241 } }
I20260807 03:07:24.730075 2033 leader_election.cc:302] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91bd4dbbe4784df28a0f6e1e4b29501b; no voters:
I20260807 03:07:24.730679 2033 leader_election.cc:288] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:24.730808 2041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:24.731444 2041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [term 1 LEADER]: Becoming Leader. State: Replica: 91bd4dbbe4784df28a0f6e1e4b29501b, State: Running, Role: LEADER
I20260807 03:07:24.731981 2041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40241 } }
I20260807 03:07:24.732592 2033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:24.734714 2043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91bd4dbbe4784df28a0f6e1e4b29501b. Latest consensus state: current_term: 1 leader_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40241 } } }
I20260807 03:07:24.735122 2043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:24.735930 2042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bd4dbbe4784df28a0f6e1e4b29501b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40241 } } }
I20260807 03:07:24.736644 2042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:24.737592 2054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:24.742648 2054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:24.750559 2054 catalog_manager.cc:1347] Generated new cluster ID: 6fea7416865041d9985bd6616258d902
I20260807 03:07:24.750669 2054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:24.760365 2054 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:24.761871 2054 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:24.772331 2054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b: Generated new TSK 0
I20260807 03:07:24.773165 2054 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
Entry for principal kudu/127.5.122.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab.
I20260807 03:07:24.843851 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40241
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:25.170670 2072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:25.170926 2072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:25.171057 2072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:25.177933 2072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:25.178051 2072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:25.178207 2072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:07:25.187225 2072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.1.keytab
--principal=kudu/127.5.122.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:25.188315 2072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:25.189512 2072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:25.201442 2078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:25.201560 2082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:25.201473 2079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:25.202505 2072 server_base.cc:1034] running on GCE node
I20260807 03:07:25.203064 2072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:25.203902 2072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:25.205091 2072 hybrid_clock.cc:648] HybridClock initialized: now 1786072045205043 us; error 41 us; skew 500 ppm
I20260807 03:07:25.205370 2072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:25 dist-test-slave-fgb3 krb5kdc[1925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:25.208400 2072 init.cc:377] Logged in from keytab as kudu/127.5.122.1@KRBTEST.COM (short username kudu)
I20260807 03:07:25.209798 2072 webserver.cc:460] Webserver started at http://127.5.122.1:40519/ using document root <none> and password file <none>
I20260807 03:07:25.210469 2072 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:25.210582 2072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:25.210865 2072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:25.213629 2072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "fc8ab070f7704e039e9b524856ec3d6d"
format_stamp: "Formatted at 2026-08-07 03:07:25 on dist-test-slave-fgb3"
server_key: "5b33c03ff1bfec75f65064d68ac26639"
server_key_iv: "f59a41eec7716768b899174c296c5426"
server_key_version: "encryptionkey@0"
I20260807 03:07:25.214406 2072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "fc8ab070f7704e039e9b524856ec3d6d"
format_stamp: "Formatted at 2026-08-07 03:07:25 on dist-test-slave-fgb3"
server_key: "5b33c03ff1bfec75f65064d68ac26639"
server_key_iv: "f59a41eec7716768b899174c296c5426"
server_key_version: "encryptionkey@0"
I20260807 03:07:25.219254 2072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 03:07:25.222470 2092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:25.223654 2072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:07:25.223803 2072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "fc8ab070f7704e039e9b524856ec3d6d"
format_stamp: "Formatted at 2026-08-07 03:07:25 on dist-test-slave-fgb3"
server_key: "5b33c03ff1bfec75f65064d68ac26639"
server_key_iv: "f59a41eec7716768b899174c296c5426"
server_key_version: "encryptionkey@0"
I20260807 03:07:25.223971 2072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:25.248720 2072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:25.252791 2072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:25.253058 2072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:25.254009 2072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:25.255352 2072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:25.255430 2072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:25.255555 2072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:25.255607 2072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:25.285560 2072 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:44663
I20260807 03:07:25.285607 2251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:44663 every 8 connection(s)
I20260807 03:07:25.286818 2072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:07:25.292927 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 2072
I20260807 03:07:25.293081 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:07:25.293542 5608 external_mini_cluster.cc:1428] Setting key 7119ea15db95c65fdc7a4efca0e84c13
Aug 07 03:07:25 dist-test-slave-fgb3 krb5kdc[1925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.1@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:25.306622 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:25.289240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:40109 (local address 127.5.122.62:40241)
0807 03:07:25.289672 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:25.289677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:25.290517 (+ 840us) server_negotiation.cc:407] Connection header received
0807 03:07:25.292302 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:25.292308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:25.292385 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:25.292491 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:25.293977 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.294995 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.297123 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.297509 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.297611 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:25.301418 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:25.301423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:25.301426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:25.301466 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:25.303830 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:25.304628 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:25.304632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:25.304634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:25.304739 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:25.305140 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:25.305142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:25.305144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:25.305366 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:25.305475 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:25.306179 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 03:07:25.306368 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":230,"threads_started":1}
I20260807 03:07:25.307833 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:25.289675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40241 (local address 127.5.122.1:40109)
0807 03:07:25.290359 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:25.290409 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:25.292053 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:25.292710 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:25.292718 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:25.293261 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:25.293818 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:25.293836 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.295164 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.295170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:25.296977 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:25.296993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.297728 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.297735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:25.298011 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:25.298708 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:25.298734 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:25.301200 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:25.303959 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:25.303968 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:25.303984 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:25.304534 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:25.304826 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:25.304830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:25.304833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:25.305055 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:25.305482 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:25.305488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:25.305843 (+ 355us) client_negotiation.cc:769] Sending connection context
0807 03:07:25.306120 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 03:07:25.306614 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":263,"threads_started":1}
I20260807 03:07:25.309224 2253 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40241
I20260807 03:07:25.309495 2253 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:25.310125 2253 heartbeater.cc:511] Master 127.5.122.62:40241 requested a full tablet report, sending...
I20260807 03:07:25.312466 1988 ts_manager.cc:194] Registered new tserver with Master: fc8ab070f7704e039e9b524856ec3d6d (127.5.122.1:44663)
I20260807 03:07:25.314530 1988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.1@KRBTEST.COM'} at 127.5.122.1:40109
WARNING: no policy specified for kudu/127.5.122.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
Entry for principal kudu/127.5.122.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab.
I20260807 03:07:25.358767 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40241
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:25.678180 2259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:25.678515 2259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:25.678660 2259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:25.685858 2259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:25.685981 2259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:25.686137 2259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:07:25.695899 2259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.2.keytab
--principal=kudu/127.5.122.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:25.697135 2259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:25.698480 2259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:25.710805 2267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:25.710877 2272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:25.710847 2270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:25.711835 2259 server_base.cc:1034] running on GCE node
I20260807 03:07:25.712435 2259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:25.713326 2259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:25.714551 2259 hybrid_clock.cc:648] HybridClock initialized: now 1786072045714496 us; error 41 us; skew 500 ppm
I20260807 03:07:25.714852 2259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:25 dist-test-slave-fgb3 krb5kdc[1925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:25.717768 2259 init.cc:377] Logged in from keytab as kudu/127.5.122.2@KRBTEST.COM (short username kudu)
I20260807 03:07:25.719239 2259 webserver.cc:460] Webserver started at http://127.5.122.2:34211/ using document root <none> and password file <none>
I20260807 03:07:25.719894 2259 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:25.720009 2259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:25.720294 2259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:25.723188 2259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "d521635e2d5c41d5bb22631412f2b96f"
format_stamp: "Formatted at 2026-08-07 03:07:25 on dist-test-slave-fgb3"
server_key: "d58b352d9bab1e7719cfa4da854f6e28"
server_key_iv: "ae0e15ae78a65a690f2a70ff9e3db482"
server_key_version: "encryptionkey@0"
I20260807 03:07:25.723987 2259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "d521635e2d5c41d5bb22631412f2b96f"
format_stamp: "Formatted at 2026-08-07 03:07:25 on dist-test-slave-fgb3"
server_key: "d58b352d9bab1e7719cfa4da854f6e28"
server_key_iv: "ae0e15ae78a65a690f2a70ff9e3db482"
server_key_version: "encryptionkey@0"
I20260807 03:07:25.729049 2259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:07:25.732349 2282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:25.733635 2259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:07:25.733790 2259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "d521635e2d5c41d5bb22631412f2b96f"
format_stamp: "Formatted at 2026-08-07 03:07:25 on dist-test-slave-fgb3"
server_key: "d58b352d9bab1e7719cfa4da854f6e28"
server_key_iv: "ae0e15ae78a65a690f2a70ff9e3db482"
server_key_version: "encryptionkey@0"
I20260807 03:07:25.733965 2259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:25.776793 2259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:25.780922 2259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:25.781227 2259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:25.782232 2259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:25.783576 2259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:25.783653 2259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:25.783775 2259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:25.783829 2259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:25.813596 2259 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:46079
I20260807 03:07:25.813634 2447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:46079 every 8 connection(s)
I20260807 03:07:25.814847 2259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:07:25.815671 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 2259
I20260807 03:07:25.815845 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:07:25.816345 5608 external_mini_cluster.cc:1428] Setting key ffa11f07b181345d33e58ef0af654402
Aug 07 03:07:25 dist-test-slave-fgb3 krb5kdc[1925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072045, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.2@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:25.834520 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:25.817283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:39713 (local address 127.5.122.62:40241)
0807 03:07:25.817724 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:25.817734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:25.818894 (+ 1160us) server_negotiation.cc:407] Connection header received
0807 03:07:25.820748 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:25.820753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:25.820840 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:25.820932 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:25.822392 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.823087 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.824584 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.825013 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.825121 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:25.828512 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:25.828518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:25.828523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:25.828559 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:25.831429 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:25.832249 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:25.832253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:25.832255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:25.832372 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:25.832902 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:25.832904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:25.832906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:25.833155 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:25.833293 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:25.834054 (+ 761us) server_negotiation.cc:299] Negotiation successful
0807 03:07:25.834221 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":213,"threads_started":1}
I20260807 03:07:25.835806 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:25.817906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40241 (local address 127.5.122.2:39713)
0807 03:07:25.818728 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:25.818765 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:25.820500 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:25.821162 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:25.821170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:25.821673 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:25.822211 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:25.822228 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.823235 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.823247 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:25.824442 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:25.824459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:25.825227 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:25.825235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:25.825448 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:25.826047 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:25.826069 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:25.828310 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:25.831558 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:25.831566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:25.831581 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:25.832151 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:25.832563 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:25.832567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:25.832570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:25.832802 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:25.833281 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:25.833289 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:25.833671 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 03:07:25.833950 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 03:07:25.834434 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":398,"threads_started":1}
I20260807 03:07:25.837230 2448 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40241
I20260807 03:07:25.837541 2448 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:25.838330 2448 heartbeater.cc:511] Master 127.5.122.62:40241 requested a full tablet report, sending...
I20260807 03:07:25.839666 1988 ts_manager.cc:194] Registered new tserver with Master: d521635e2d5c41d5bb22631412f2b96f (127.5.122.2:46079)
I20260807 03:07:25.840440 1988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.2@KRBTEST.COM'} at 127.5.122.2:39713
WARNING: no policy specified for kudu/127.5.122.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
Entry for principal kudu/127.5.122.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab.
I20260807 03:07:25.879669 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40241
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:26.203146 2456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:26.203418 2456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:26.203552 2456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:26.210531 2456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:26.210647 2456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:26.210791 2456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:07:26.220206 2456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.3.keytab
--principal=kudu/127.5.122.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:40241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:26.221438 2456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:26.222714 2456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:26.234997 2466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:26.235054 2463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:26.235030 2464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:26.235987 2456 server_base.cc:1034] running on GCE node
I20260807 03:07:26.236564 2456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:26.237439 2456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:26.238656 2456 hybrid_clock.cc:648] HybridClock initialized: now 1786072046238610 us; error 30 us; skew 500 ppm
I20260807 03:07:26.238947 2456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:26 dist-test-slave-fgb3 krb5kdc[1925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:26.241916 2456 init.cc:377] Logged in from keytab as kudu/127.5.122.3@KRBTEST.COM (short username kudu)
I20260807 03:07:26.243322 2456 webserver.cc:460] Webserver started at http://127.5.122.3:35389/ using document root <none> and password file <none>
I20260807 03:07:26.243937 2456 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:26.244045 2456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:26.244333 2456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:26.247084 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "257b2ad7b8fd4b8d890837a9b2598227"
format_stamp: "Formatted at 2026-08-07 03:07:26 on dist-test-slave-fgb3"
server_key: "088ccb6616c26ac1072a621d28fef3e2"
server_key_iv: "864bd137afb73273fe4d65e26e5d7350"
server_key_version: "encryptionkey@0"
I20260807 03:07:26.247825 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "257b2ad7b8fd4b8d890837a9b2598227"
format_stamp: "Formatted at 2026-08-07 03:07:26 on dist-test-slave-fgb3"
server_key: "088ccb6616c26ac1072a621d28fef3e2"
server_key_iv: "864bd137afb73273fe4d65e26e5d7350"
server_key_version: "encryptionkey@0"
I20260807 03:07:26.252570 2456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 03:07:26.255764 2475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:26.256932 2456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 03:07:26.257078 2456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "257b2ad7b8fd4b8d890837a9b2598227"
format_stamp: "Formatted at 2026-08-07 03:07:26 on dist-test-slave-fgb3"
server_key: "088ccb6616c26ac1072a621d28fef3e2"
server_key_iv: "864bd137afb73273fe4d65e26e5d7350"
server_key_version: "encryptionkey@0"
I20260807 03:07:26.257244 2456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:26.270109 2456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:26.273984 2456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:26.274225 2456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:26.275089 2456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:26.276263 2456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:26.276343 2456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:26.276455 2456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:26.276508 2456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:26.306479 2456 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:33201
I20260807 03:07:26.306510 2630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:33201 every 8 connection(s)
I20260807 03:07:26.307636 2456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:07:26.312644 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 2456
I20260807 03:07:26.312788 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:07:26.313344 5608 external_mini_cluster.cc:1428] Setting key 22a6e14c3ce840eb2d00483702d4d9c8
I20260807 03:07:26.318434 2253 heartbeater.cc:503] Master 127.5.122.62:40241 was elected leader, sending a full tablet report...
Aug 07 03:07:26 dist-test-slave-fgb3 krb5kdc[1925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.3@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:26.325700 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.309916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:53169 (local address 127.5.122.62:40241)
0807 03:07:26.310124 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.310130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.311174 (+ 1044us) server_negotiation.cc:407] Connection header received
0807 03:07:26.312951 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.312957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.313043 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.313155 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:26.314786 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.315749 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.317374 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.317761 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.317854 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.321023 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:26.321027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:26.321031 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:26.321061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:26.323353 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.323986 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.323990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.323992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.324098 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.324429 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.324432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.324434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.324678 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:26.324781 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.325322 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.325488 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 03:07:26.326679 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.310354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:40241 (local address 127.5.122.3:53169)
0807 03:07:26.311024 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:26.311058 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:26.312676 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:26.313313 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:26.313319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:26.313888 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:26.314615 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.314639 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.315900 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.315905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.317239 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.317252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.317919 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.317923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.318113 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.318780 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:07:26.318800 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:26.320852 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:26.323479 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:26.323486 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:26.323496 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:26.323899 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:26.324168 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:07:26.324171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:07:26.324173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:07:26.324352 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:07:26.324763 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:26.324768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:26.325041 (+ 273us) client_negotiation.cc:769] Sending connection context
0807 03:07:26.325238 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 03:07:26.325595 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":259,"threads_started":1}
I20260807 03:07:26.328150 2632 heartbeater.cc:347] Connected to a master server at 127.5.122.62:40241
I20260807 03:07:26.328451 2632 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:26.329043 2632 heartbeater.cc:511] Master 127.5.122.62:40241 requested a full tablet report, sending...
I20260807 03:07:26.330237 1988 ts_manager.cc:194] Registered new tserver with Master: 257b2ad7b8fd4b8d890837a9b2598227 (127.5.122.3:33201)
I20260807 03:07:26.330981 1988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.3@KRBTEST.COM'} at 127.5.122.3:53169
I20260807 03:07:26.340108 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:07:26.355099 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.344029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.5.122.62:40241)
0807 03:07:26.344167 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.344171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.344603 (+ 432us) server_negotiation.cc:407] Connection header received
0807 03:07:26.344739 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.344742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.344803 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.344868 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:26.346129 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.346848 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.348051 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.348418 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.348504 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.350635 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:26.350639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:26.350642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:26.350666 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:26.352907 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.353491 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.353494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.353496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.353603 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.353872 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.353875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.353877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.354138 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:26.354347 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.354713 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.354860 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 03:07:26.358577 1988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:07:26.365483 1988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:07:26.389223 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.377495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44663 (local address 127.0.0.1:47046)
0807 03:07:26.379551 (+ 2056us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:26.379570 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:26.379699 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:26.380760 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:26.380768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:26.380820 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:26.381216 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.381229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.383424 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.383430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.385833 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.385850 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.388216 (+ 2366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.388223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.388435 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.388488 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 03:07:26.388625 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 03:07:26.388840 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1925,"thread_start_us":3233,"threads_started":1}
I20260807 03:07:26.390205 2651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.377918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47046 (local address 127.5.122.1:44663)
0807 03:07:26.379108 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.379116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.379835 (+ 719us) server_negotiation.cc:407] Connection header received
0807 03:07:26.379969 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.379978 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.380238 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.380431 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:26.381436 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.383234 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.386032 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.387656 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.387801 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.388055 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.389650 (+ 1595us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.389851 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1013,"thread_start_us":823,"threads_started":1}
I20260807 03:07:26.395370 2164 tablet_service.cc:1467] Processing CreateTablet for tablet 091e83e7aefc420ab33cf0445457ba99 (DEFAULT_TABLE table=test-table [id=ec455ef922ef4907a32929e413a80829]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:26.396673 2164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 091e83e7aefc420ab33cf0445457ba99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:26.398422 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.377430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.5.122.2:46079)
0807 03:07:26.391346 (+ 13916us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.391354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.391384 (+ 30us) server_negotiation.cc:407] Connection header received
0807 03:07:26.391483 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.391492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.391771 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.392000 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:26.392491 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.393692 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.396109 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.397388 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.397502 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.397700 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.397923 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.398114 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13766,"thread_start_us":13658,"threads_started":1}
I20260807 03:07:26.398432 2646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.377211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:46079 (local address 127.0.0.1:55908)
0807 03:07:26.378188 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:26.378245 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:26.378448 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:26.391996 (+ 13548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:26.392001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:26.392027 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:26.392331 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.392341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.393846 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.393851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.395915 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.395930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.397581 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.397587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.397748 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.397781 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 03:07:26.397861 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 03:07:26.398000 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":25,"thread_start_us":509,"threads_started":1}
I20260807 03:07:26.399590 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.379186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:33201 (local address 127.0.0.1:60696)
0807 03:07:26.390717 (+ 11531us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:26.390737 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:26.390864 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:26.392888 (+ 2024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:26.392893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:26.392918 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:26.393101 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.393110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.394995 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.395001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.396857 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.396871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.398835 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.398841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.399008 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.399040 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:07:26.399139 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 03:07:26.399270 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11394,"mutex_wait_us":22,"thread_start_us":12411,"threads_started":1}
I20260807 03:07:26.399778 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.379510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.5.122.3:33201)
0807 03:07:26.392341 (+ 12831us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.392350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.392384 (+ 34us) server_negotiation.cc:407] Connection header received
0807 03:07:26.392482 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.392489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.392696 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.392888 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:26.393263 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.394836 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.397018 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.398407 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.398526 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.398737 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.399209 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.399366 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12659,"thread_start_us":12297,"threads_started":1}
I20260807 03:07:26.402076 2345 tablet_service.cc:1467] Processing CreateTablet for tablet 091e83e7aefc420ab33cf0445457ba99 (DEFAULT_TABLE table=test-table [id=ec455ef922ef4907a32929e413a80829]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:26.403285 2345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 091e83e7aefc420ab33cf0445457ba99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:26.402772 2541 tablet_service.cc:1467] Processing CreateTablet for tablet 091e83e7aefc420ab33cf0445457ba99 (DEFAULT_TABLE table=test-table [id=ec455ef922ef4907a32929e413a80829]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:26.404161 2541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 091e83e7aefc420ab33cf0445457ba99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:26.413746 2654 tablet_bootstrap.cc:492] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Bootstrap starting.
I20260807 03:07:26.415014 2655 tablet_bootstrap.cc:492] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Bootstrap starting.
I20260807 03:07:26.418207 2656 tablet_bootstrap.cc:492] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Bootstrap starting.
I20260807 03:07:26.418812 2655 tablet_bootstrap.cc:654] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:26.418812 2654 tablet_bootstrap.cc:654] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:26.420110 2654 log.cc:824] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:26.420110 2655 log.cc:824] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:26.421387 2656 tablet_bootstrap.cc:654] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:26.422591 2656 log.cc:824] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:26.423450 2655 tablet_bootstrap.cc:492] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: No bootstrap required, opened a new log
I20260807 03:07:26.423763 2655 ts_tablet_manager.cc:1397] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 03:07:26.424067 2654 tablet_bootstrap.cc:492] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: No bootstrap required, opened a new log
I20260807 03:07:26.424328 2654 ts_tablet_manager.cc:1397] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 03:07:26.424981 2656 tablet_bootstrap.cc:492] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: No bootstrap required, opened a new log
I20260807 03:07:26.425205 2656 ts_tablet_manager.cc:1397] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 03:07:26.432832 2656 raft_consensus.cc:348] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.433174 2656 raft_consensus.cc:374] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:26.433302 2656 raft_consensus.cc:729] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257b2ad7b8fd4b8d890837a9b2598227, State: Initialized, Role: FOLLOWER
I20260807 03:07:26.433884 2656 consensus_queue.cc:260] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.434453 2655 raft_consensus.cc:348] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.434463 2654 raft_consensus.cc:348] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.434854 2654 raft_consensus.cc:374] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:26.434854 2655 raft_consensus.cc:374] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:26.434966 2654 raft_consensus.cc:729] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d521635e2d5c41d5bb22631412f2b96f, State: Initialized, Role: FOLLOWER
I20260807 03:07:26.434962 2655 raft_consensus.cc:729] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8ab070f7704e039e9b524856ec3d6d, State: Initialized, Role: FOLLOWER
I20260807 03:07:26.435509 2654 consensus_queue.cc:260] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.435503 2655 consensus_queue.cc:260] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.436673 2448 heartbeater.cc:503] Master 127.5.122.62:40241 was elected leader, sending a full tablet report...
I20260807 03:07:26.437409 2654 ts_tablet_manager.cc:1428] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260807 03:07:26.439129 2655 ts_tablet_manager.cc:1428] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
I20260807 03:07:26.440882 2632 heartbeater.cc:503] Master 127.5.122.62:40241 was elected leader, sending a full tablet report...
I20260807 03:07:26.441432 2656 ts_tablet_manager.cc:1428] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
W20260807 03:07:26.538996 2254 tablet.cc:2369] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:07:26.559187 2633 tablet.cc:2369] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:07:26.566743 2449 tablet.cc:2369] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:07:26.569604 2660 raft_consensus.cc:482] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:26.569857 2660 raft_consensus.cc:504] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.571452 2660 leader_election.cc:288] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d521635e2d5c41d5bb22631412f2b96f (127.5.122.2:46079), fc8ab070f7704e039e9b524856ec3d6d (127.5.122.1:44663)
I20260807 03:07:26.577446 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.571691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:46079 (local address 127.5.122.3:55719)
0807 03:07:26.571884 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:26.571903 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:26.572019 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:26.572411 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:26.572420 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:26.572452 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:26.572652 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.572665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.573807 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.573811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.575659 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.575675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.576935 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.576939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.577074 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.577100 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 03:07:26.577167 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 03:07:26.577254 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 03:07:26.577597 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.571855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:55719 (local address 127.5.122.2:46079)
0807 03:07:26.572053 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.572059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.572080 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:07:26.572159 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.572164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.572248 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.572371 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:26.572768 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.573669 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.575803 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.576790 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.576919 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.577016 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.577211 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.577349 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:07:26.578400 2651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.572041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:34245 (local address 127.5.122.1:44663)
0807 03:07:26.572256 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.572261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.572684 (+ 423us) server_negotiation.cc:407] Connection header received
0807 03:07:26.572853 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.572858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.572930 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.573024 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:26.573453 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.574218 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.576300 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.577446 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.577569 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.577663 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.578109 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.578206 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 03:07:26.578557 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.571715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:44663 (local address 127.5.122.3:34245)
0807 03:07:26.572578 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:26.572598 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:26.572710 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:26.573037 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:26.573040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:26.573059 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:26.573321 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.573328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.574397 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.574403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.576156 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:26.576167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.577814 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.577820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:26.577943 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.577967 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 03:07:26.578052 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 03:07:26.578158 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":218,"thread_start_us":304,"threads_started":1}
I20260807 03:07:26.578666 2373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091e83e7aefc420ab33cf0445457ba99" candidate_uuid: "257b2ad7b8fd4b8d890837a9b2598227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d521635e2d5c41d5bb22631412f2b96f" is_pre_election: true
I20260807 03:07:26.579177 2373 raft_consensus.cc:2454] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 257b2ad7b8fd4b8d890837a9b2598227 in term 0.
I20260807 03:07:26.579387 2193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091e83e7aefc420ab33cf0445457ba99" candidate_uuid: "257b2ad7b8fd4b8d890837a9b2598227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8ab070f7704e039e9b524856ec3d6d" is_pre_election: true
I20260807 03:07:26.579802 2476 leader_election.cc:302] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257b2ad7b8fd4b8d890837a9b2598227, d521635e2d5c41d5bb22631412f2b96f; no voters:
I20260807 03:07:26.579897 2193 raft_consensus.cc:2454] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 257b2ad7b8fd4b8d890837a9b2598227 in term 0.
I20260807 03:07:26.580199 2660 raft_consensus.cc:2781] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:07:26.580343 2660 raft_consensus.cc:482] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:07:26.580462 2660 raft_consensus.cc:3037] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:26.582656 2660 raft_consensus.cc:504] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.583223 2660 leader_election.cc:288] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [CANDIDATE]: Term 1 election: Requested vote from peers d521635e2d5c41d5bb22631412f2b96f (127.5.122.2:46079), fc8ab070f7704e039e9b524856ec3d6d (127.5.122.1:44663)
I20260807 03:07:26.583598 2373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091e83e7aefc420ab33cf0445457ba99" candidate_uuid: "257b2ad7b8fd4b8d890837a9b2598227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d521635e2d5c41d5bb22631412f2b96f"
I20260807 03:07:26.583631 2193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "091e83e7aefc420ab33cf0445457ba99" candidate_uuid: "257b2ad7b8fd4b8d890837a9b2598227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8ab070f7704e039e9b524856ec3d6d"
I20260807 03:07:26.583779 2373 raft_consensus.cc:3037] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:26.583787 2193 raft_consensus.cc:3037] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:26.585917 2373 raft_consensus.cc:2454] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257b2ad7b8fd4b8d890837a9b2598227 in term 1.
I20260807 03:07:26.586018 2193 raft_consensus.cc:2454] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257b2ad7b8fd4b8d890837a9b2598227 in term 1.
I20260807 03:07:26.586277 2476 leader_election.cc:302] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257b2ad7b8fd4b8d890837a9b2598227, d521635e2d5c41d5bb22631412f2b96f; no voters:
I20260807 03:07:26.586570 2660 raft_consensus.cc:2781] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:26.586985 2660 raft_consensus.cc:686] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 1 LEADER]: Becoming Leader. State: Replica: 257b2ad7b8fd4b8d890837a9b2598227, State: Running, Role: LEADER
I20260807 03:07:26.587488 2660 consensus_queue.cc:237] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } }
I20260807 03:07:26.591784 1986 catalog_manager.cc:5495] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 257b2ad7b8fd4b8d890837a9b2598227 (127.5.122.3). New cstate: current_term: 1 leader_uuid: "257b2ad7b8fd4b8d890837a9b2598227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "257b2ad7b8fd4b8d890837a9b2598227" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 33201 } health_report { overall_health: HEALTHY } } }
I20260807 03:07:26.624245 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.619260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60704 (local address 127.5.122.3:33201)
0807 03:07:26.619412 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.619417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.619567 (+ 150us) server_negotiation.cc:407] Connection header received
0807 03:07:26.619758 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.619762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.619834 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.619953 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:07:26.620317 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.621030 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.622423 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.622806 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.622891 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.623327 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:07:26.623789 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:07:26.623892 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.624000 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.624060 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 03:07:26.632179 2373 raft_consensus.cc:1261] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 1 FOLLOWER]: Refusing update from remote peer 257b2ad7b8fd4b8d890837a9b2598227: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:26.632179 2193 raft_consensus.cc:1261] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 1 FOLLOWER]: Refusing update from remote peer 257b2ad7b8fd4b8d890837a9b2598227: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:26.633188 2664 consensus_queue.cc:1035] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d521635e2d5c41d5bb22631412f2b96f" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 46079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:26.633594 2660 consensus_queue.cc:1035] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8ab070f7704e039e9b524856ec3d6d" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:26.647629 2672 mvcc.cc:205] Tried to move back new op lower bound from 7315751102995083264 to 7315751102826135552. Current Snapshot: MvccSnapshot[applied={T|T < 7315751102995083264}]
I20260807 03:07:26.659821 1986 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:26.660109 1986 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:26.663610 1986 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91bd4dbbe4784df28a0f6e1e4b29501b: Sending DeleteTablet for 3 replicas of tablet 091e83e7aefc420ab33cf0445457ba99
I20260807 03:07:26.664743 2345 tablet_service.cc:1514] Processing DeleteTablet for tablet 091e83e7aefc420ab33cf0445457ba99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:26 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:55908
I20260807 03:07:26.664881 2164 tablet_service.cc:1514] Processing DeleteTablet for tablet 091e83e7aefc420ab33cf0445457ba99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:26 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:47046
I20260807 03:07:26.665524 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 2072
I20260807 03:07:26.666455 2541 tablet_service.cc:1514] Processing DeleteTablet for tablet 091e83e7aefc420ab33cf0445457ba99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:26 UTC) from {username='kudu', principal='kudu/127.5.122.62@KRBTEST.COM'} at 127.0.0.1:60696
I20260807 03:07:26.675501 2684 tablet_replica.cc:330] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: stopping tablet replica
I20260807 03:07:26.676462 2684 raft_consensus.cc:2229] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:26.677435 2684 raft_consensus.cc:2258] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:26.678728 2683 tablet_replica.cc:330] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: stopping tablet replica
I20260807 03:07:26.679709 2685 tablet_replica.cc:330] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: stopping tablet replica
I20260807 03:07:26.680322 2685 raft_consensus.cc:2229] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:07:26.680681 2684 ts_tablet_manager.cc:1905] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:26.681226 2685 raft_consensus.cc:2258] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:26.683193 2685 ts_tablet_manager.cc:1905] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:26.687656 2684 ts_tablet_manager.cc:1918] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:26.688119 2683 raft_consensus.cc:2229] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:26.688614 2683 raft_consensus.cc:2258] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:26.689093 2684 log.cc:1196] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-0/wal/wals/091e83e7aefc420ab33cf0445457ba99
I20260807 03:07:26.690085 2684 ts_tablet_manager.cc:1939] T 091e83e7aefc420ab33cf0445457ba99 P fc8ab070f7704e039e9b524856ec3d6d: Deleting consensus metadata
I20260807 03:07:26.690428 2683 ts_tablet_manager.cc:1905] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:26.692241 1959 catalog_manager.cc:4841] TS fc8ab070f7704e039e9b524856ec3d6d (127.5.122.1:44663): tablet 091e83e7aefc420ab33cf0445457ba99 (table test-table [id=ec455ef922ef4907a32929e413a80829]) successfully deleted
I20260807 03:07:26.695504 2683 ts_tablet_manager.cc:1918] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:26.695622 2683 log.cc:1196] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/091e83e7aefc420ab33cf0445457ba99
I20260807 03:07:26.695747 2685 ts_tablet_manager.cc:1918] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:26.695904 2685 log.cc:1196] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/091e83e7aefc420ab33cf0445457ba99
I20260807 03:07:26.696331 2683 ts_tablet_manager.cc:1939] T 091e83e7aefc420ab33cf0445457ba99 P d521635e2d5c41d5bb22631412f2b96f: Deleting consensus metadata
I20260807 03:07:26.696516 2685 ts_tablet_manager.cc:1939] T 091e83e7aefc420ab33cf0445457ba99 P 257b2ad7b8fd4b8d890837a9b2598227: Deleting consensus metadata
I20260807 03:07:26.698079 1958 catalog_manager.cc:4841] TS d521635e2d5c41d5bb22631412f2b96f (127.5.122.2:46079): tablet 091e83e7aefc420ab33cf0445457ba99 (table test-table [id=ec455ef922ef4907a32929e413a80829]) successfully deleted
I20260807 03:07:26.698108 1961 catalog_manager.cc:4841] TS 257b2ad7b8fd4b8d890837a9b2598227 (127.5.122.3:33201): tablet 091e83e7aefc420ab33cf0445457ba99 (table test-table [id=ec455ef922ef4907a32929e413a80829]) successfully deleted
Aug 07 03:07:26 dist-test-slave-fgb3 krb5kdc[1925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.1@KRBTEST.COM
I20260807 03:07:26.708344 2651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.666634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.5.122.1:44663)
0807 03:07:26.666815 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.666821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.681698 (+ 14877us) server_negotiation.cc:407] Connection header received
0807 03:07:26.682469 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.682475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.682570 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.682658 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:26.684638 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.685811 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.687755 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.688401 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.688506 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.701083 (+ 12577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:26.701100 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:26.701117 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:26.701160 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:26.705043 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.705868 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.705878 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.705881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.706042 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.706500 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.706506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.706510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.707008 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:26.707318 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.707622 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.707842 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:07:26.709092 2246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:26.797472 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 2072
I20260807 03:07:26.828269 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 2259
Aug 07 03:07:26 dist-test-slave-fgb3 krb5kdc[1925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.2@KRBTEST.COM
I20260807 03:07:26.845448 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.829109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.5.122.2:46079)
0807 03:07:26.829349 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.829355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.829427 (+ 72us) server_negotiation.cc:407] Connection header received
0807 03:07:26.829608 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.829614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.829707 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.829810 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:26.831524 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.832288 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.834102 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.834555 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.834649 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.839918 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:26.839931 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:26.839947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:26.839985 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:26.842773 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.843559 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.843566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.843568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.843696 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.844059 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.844062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.844064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.844459 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:26.844679 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.844986 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.845156 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 03:07:26.846264 2437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:26.933972 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 2259
I20260807 03:07:26.957561 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 2456
Aug 07 03:07:26 dist-test-slave-fgb3 krb5kdc[1925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.3@KRBTEST.COM
I20260807 03:07:26.975422 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:26.958431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.5.122.3:33201)
0807 03:07:26.958589 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:26.958593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:26.958610 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:07:26.958743 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:26.958747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:26.958819 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:26.958895 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:26.960794 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.961695 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.963539 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:26.964037 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:26.964135 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:26.969564 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:26.969582 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:26.969598 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:26.969641 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:26.972478 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.973312 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.973319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.973322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.973497 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:26.973917 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:26.973921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:26.973923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:26.974344 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:26.974592 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:26.974913 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 03:07:26.975090 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:07:26.976200 2626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:27.097291 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 2456
I20260807 03:07:27.119354 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 1939
I20260807 03:07:27.120491 2027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:27.228638 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 1939
2026-08-07T03:07:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4440 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T03:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:27Z Disabled control of system clock
I20260807 03:07:27.318853 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:35097
--webserver_interface=127.5.122.62
--webserver_port=0
--builtin_ntp_servers=127.5.122.20:34657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 03:07:27.646273 2699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:27.646602 2699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:27.646746 2699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:27.653918 2699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:27.654050 2699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:27.654124 2699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:27.654211 2699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:27.654286 2699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:27.663518 2699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:34657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:35097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:27.664860 2699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:27.666831 2699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:27.678372 2714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:27.678491 2709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:27.678488 2710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:27.679298 2699 server_base.cc:1034] running on GCE node
I20260807 03:07:27.680070 2699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:27.681457 2699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:27.682664 2699 hybrid_clock.cc:648] HybridClock initialized: now 1786072047682619 us; error 35 us; skew 500 ppm
I20260807 03:07:27.683014 2699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:27.685146 2699 webserver.cc:460] Webserver started at http://127.5.122.62:46145/ using document root <none> and password file <none>
I20260807 03:07:27.685808 2699 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:27.685905 2699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:27.686208 2699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:27.689024 2699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "8de297c13af64b99a4304a67ad9fe62b"
format_stamp: "Formatted at 2026-08-07 03:07:27 on dist-test-slave-fgb3"
server_key: "5ff3d87fab8f80f4948d2a32bfb6f1a0"
server_key_iv: "21f8b48bcfe3543cfdce65b2914c501b"
server_key_version: "encryptionkey@0"
I20260807 03:07:27.689841 2699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "8de297c13af64b99a4304a67ad9fe62b"
format_stamp: "Formatted at 2026-08-07 03:07:27 on dist-test-slave-fgb3"
server_key: "5ff3d87fab8f80f4948d2a32bfb6f1a0"
server_key_iv: "21f8b48bcfe3543cfdce65b2914c501b"
server_key_version: "encryptionkey@0"
I20260807 03:07:27.694691 2699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 03:07:27.697999 2722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:27.699311 2699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:07:27.699477 2699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "8de297c13af64b99a4304a67ad9fe62b"
format_stamp: "Formatted at 2026-08-07 03:07:27 on dist-test-slave-fgb3"
server_key: "5ff3d87fab8f80f4948d2a32bfb6f1a0"
server_key_iv: "21f8b48bcfe3543cfdce65b2914c501b"
server_key_version: "encryptionkey@0"
I20260807 03:07:27.699668 2699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:27.718268 2699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:27.719028 2699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:27.719295 2699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:27.736052 2699 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:35097
I20260807 03:07:27.736086 2790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:35097 every 8 connection(s)
I20260807 03:07:27.737214 2699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:27.740643 2791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:27.744161 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 2699
I20260807 03:07:27.744346 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:27.744894 5608 external_mini_cluster.cc:1428] Setting key 75d9f25581a5aadebea70018959cdb8a
I20260807 03:07:27.752458 2791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b: Bootstrap starting.
I20260807 03:07:27.755389 2791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:27.756461 2791 log.cc:824] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:27.757329 2794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:27.746714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54110 (local address 127.5.122.62:35097)
0807 03:07:27.747624 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:27.747634 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:27.747672 (+ 38us) server_negotiation.cc:407] Connection header received
0807 03:07:27.749252 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:27.749273 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:27.749697 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:27.750040 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:27.750618 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:27.751864 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:27.753168 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:27.753849 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:27.754020 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:27.755118 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:27.755126 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:27.755138 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:27.755664 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:27.755776 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:27.755806 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:27.755908 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:27.756132 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 03:07:27.756417 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":332,"threads_started":1}
I20260807 03:07:27.759403 2791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b: No bootstrap required, opened a new log
I20260807 03:07:27.767020 2791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de297c13af64b99a4304a67ad9fe62b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 35097 } }
I20260807 03:07:27.767360 2791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:27.767484 2791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de297c13af64b99a4304a67ad9fe62b, State: Initialized, Role: FOLLOWER
I20260807 03:07:27.768088 2791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de297c13af64b99a4304a67ad9fe62b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 35097 } }
I20260807 03:07:27.768299 2791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:27.768397 2791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:27.768514 2791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:27.770920 2791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de297c13af64b99a4304a67ad9fe62b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 35097 } }
I20260807 03:07:27.771420 2791 leader_election.cc:302] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8de297c13af64b99a4304a67ad9fe62b; no voters:
I20260807 03:07:27.771977 2791 leader_election.cc:288] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:27.772112 2798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:27.772815 2798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [term 1 LEADER]: Becoming Leader. State: Replica: 8de297c13af64b99a4304a67ad9fe62b, State: Running, Role: LEADER
I20260807 03:07:27.773268 2798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de297c13af64b99a4304a67ad9fe62b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 35097 } }
I20260807 03:07:27.773913 2791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:27.775944 2800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8de297c13af64b99a4304a67ad9fe62b. Latest consensus state: current_term: 1 leader_uuid: "8de297c13af64b99a4304a67ad9fe62b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de297c13af64b99a4304a67ad9fe62b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 35097 } } }
I20260807 03:07:27.776351 2800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:27.777256 2799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8de297c13af64b99a4304a67ad9fe62b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de297c13af64b99a4304a67ad9fe62b" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 35097 } } }
I20260807 03:07:27.777595 2799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:27.778900 2811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:27.782989 2811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:27.791077 2811 catalog_manager.cc:1347] Generated new cluster ID: 1957d511ae264a80a1574ae560a4f215
I20260807 03:07:27.791189 2811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:27.823773 2811 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:27.825573 2811 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:27.845054 2811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b: Generated new TSK 0
I20260807 03:07:27.845885 2811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:27.863682 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.1:0
--local_ip_for_outbound_sockets=127.5.122.1
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:35097
--builtin_ntp_servers=127.5.122.20:34657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 03:07:28.216595 2824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:28.216856 2824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:28.216980 2824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:28.224303 2824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:28.224390 2824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:28.224547 2824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.1
I20260807 03:07:28.233137 2824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:34657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.1
--webserver_port=0
--tserver_master_addrs=127.5.122.62:35097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.1
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:28.234339 2824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:28.235988 2824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:28.247880 2831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:28.247927 2832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:28.247951 2835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:28.248895 2824 server_base.cc:1034] running on GCE node
I20260807 03:07:28.249420 2824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:28.250252 2824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:28.251463 2824 hybrid_clock.cc:648] HybridClock initialized: now 1786072048251410 us; error 46 us; skew 500 ppm
I20260807 03:07:28.251734 2824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:28.253715 2824 webserver.cc:460] Webserver started at http://127.5.122.1:36445/ using document root <none> and password file <none>
I20260807 03:07:28.254380 2824 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:28.254485 2824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:28.254774 2824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:28.257416 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data/instance:
uuid: "b0602f36d77b4594914a48d7cb3d80ca"
format_stamp: "Formatted at 2026-08-07 03:07:28 on dist-test-slave-fgb3"
server_key: "ff4ca7421f782f0b06a170c2fe9810c3"
server_key_iv: "a8b872f2252d456f58869f48f73a5585"
server_key_version: "encryptionkey@0"
I20260807 03:07:28.258248 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance:
uuid: "b0602f36d77b4594914a48d7cb3d80ca"
format_stamp: "Formatted at 2026-08-07 03:07:28 on dist-test-slave-fgb3"
server_key: "ff4ca7421f782f0b06a170c2fe9810c3"
server_key_iv: "a8b872f2252d456f58869f48f73a5585"
server_key_version: "encryptionkey@0"
I20260807 03:07:28.263072 2824 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 03:07:28.266162 2843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:28.267463 2824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:07:28.267590 2824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/wal
uuid: "b0602f36d77b4594914a48d7cb3d80ca"
format_stamp: "Formatted at 2026-08-07 03:07:28 on dist-test-slave-fgb3"
server_key: "ff4ca7421f782f0b06a170c2fe9810c3"
server_key_iv: "a8b872f2252d456f58869f48f73a5585"
server_key_version: "encryptionkey@0"
I20260807 03:07:28.267758 2824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:28.281536 2824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:28.282178 2824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:28.282435 2824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:28.283291 2824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:28.284590 2824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:28.284672 2824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:28.284750 2824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:28.284802 2824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:28.312147 2824 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.1:40191
I20260807 03:07:28.312202 3008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.1:40191 every 8 connection(s)
I20260807 03:07:28.313409 2824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/data/info.pb
I20260807 03:07:28.318826 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 2824
I20260807 03:07:28.318969 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-0/wal/instance
I20260807 03:07:28.319378 5608 external_mini_cluster.cc:1428] Setting key d5668d68355205212c8b5ae8d4b23ae9
I20260807 03:07:28.321950 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.2:0
--local_ip_for_outbound_sockets=127.5.122.2
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:35097
--builtin_ntp_servers=127.5.122.20:34657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 03:07:28.325611 3011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:28.315870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.1:47079 (local address 127.5.122.62:35097)
0807 03:07:28.316308 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:28.316312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:28.317325 (+ 1013us) server_negotiation.cc:407] Connection header received
0807 03:07:28.319171 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:28.319177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:28.319265 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:28.319361 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:28.320516 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.321485 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.323052 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.323439 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.323549 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:28.324804 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:28.324809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:28.324812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:28.324830 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:28.324872 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:28.324884 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:28.324958 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:28.325166 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 03:07:28.325262 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":324,"thread_start_us":223,"threads_started":1}
I20260807 03:07:28.326583 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:28.316291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:35097 (local address 127.5.122.1:47079)
0807 03:07:28.317165 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:28.317200 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:28.318889 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:28.319602 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:28.319610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:28.319691 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:28.320349 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:28.320367 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.321626 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.321630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:28.322918 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:28.322931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.323651 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.323658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:28.323874 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:28.324508 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:28.324511 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:28.324553 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:28.324625 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:28.324974 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:28.324985 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:28.324993 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 03:07:28.325141 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 03:07:28.325525 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":286,"threads_started":1}
I20260807 03:07:28.328015 3009 heartbeater.cc:347] Connected to a master server at 127.5.122.62:35097
I20260807 03:07:28.328287 3009 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:28.328997 3009 heartbeater.cc:511] Master 127.5.122.62:35097 requested a full tablet report, sending...
I20260807 03:07:28.331049 2745 ts_manager.cc:194] Registered new tserver with Master: b0602f36d77b4594914a48d7cb3d80ca (127.5.122.1:40191)
I20260807 03:07:28.332835 2745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.1:47079
W20260807 03:07:28.643440 3014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:28.643672 3014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:28.643761 3014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:28.650518 3014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:28.650599 3014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:28.650701 3014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.2
I20260807 03:07:28.659175 3014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:34657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.2
--webserver_port=0
--tserver_master_addrs=127.5.122.62:35097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.2
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:28.660188 3014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:28.661775 3014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:28.673632 3028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:28.673651 3024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:28.673683 3025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:28.674431 3014 server_base.cc:1034] running on GCE node
I20260807 03:07:28.675015 3014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:28.675863 3014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:28.677050 3014 hybrid_clock.cc:648] HybridClock initialized: now 1786072048677008 us; error 31 us; skew 500 ppm
I20260807 03:07:28.677323 3014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:28.679371 3014 webserver.cc:460] Webserver started at http://127.5.122.2:43167/ using document root <none> and password file <none>
I20260807 03:07:28.679977 3014 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:28.680074 3014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:28.680363 3014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:28.683053 3014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data/instance:
uuid: "04f6488bcd934c62999c813f53040a07"
format_stamp: "Formatted at 2026-08-07 03:07:28 on dist-test-slave-fgb3"
server_key: "b8a963e98c921f624727e7dbdf190a7f"
server_key_iv: "84efc2b6e5a6ae4eccc4c1b1b83cd20a"
server_key_version: "encryptionkey@0"
I20260807 03:07:28.683784 3014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance:
uuid: "04f6488bcd934c62999c813f53040a07"
format_stamp: "Formatted at 2026-08-07 03:07:28 on dist-test-slave-fgb3"
server_key: "b8a963e98c921f624727e7dbdf190a7f"
server_key_iv: "84efc2b6e5a6ae4eccc4c1b1b83cd20a"
server_key_version: "encryptionkey@0"
I20260807 03:07:28.688271 3014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 03:07:28.691345 3034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:28.692489 3014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 03:07:28.692617 3014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal
uuid: "04f6488bcd934c62999c813f53040a07"
format_stamp: "Formatted at 2026-08-07 03:07:28 on dist-test-slave-fgb3"
server_key: "b8a963e98c921f624727e7dbdf190a7f"
server_key_iv: "84efc2b6e5a6ae4eccc4c1b1b83cd20a"
server_key_version: "encryptionkey@0"
I20260807 03:07:28.692796 3014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:28.708340 3014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:28.709038 3014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:28.709282 3014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:28.710188 3014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:28.711601 3014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:28.711714 3014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:28.711824 3014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:28.711896 3014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:28.742655 3014 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.2:36935
I20260807 03:07:28.742702 3194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.2:36935 every 8 connection(s)
I20260807 03:07:28.743880 3014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/data/info.pb
I20260807 03:07:28.752637 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 3014
I20260807 03:07:28.752816 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal/instance
I20260807 03:07:28.753326 5608 external_mini_cluster.cc:1428] Setting key 928349c3a6b835486d0dcdf1f5332055
I20260807 03:07:28.756242 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.3:0
--local_ip_for_outbound_sockets=127.5.122.3
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:35097
--builtin_ntp_servers=127.5.122.20:34657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 03:07:28.757690 3011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:28.746664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:57491 (local address 127.5.122.62:35097)
0807 03:07:28.746862 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:28.746868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:28.748333 (+ 1465us) server_negotiation.cc:407] Connection header received
0807 03:07:28.750769 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:28.750773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:28.750837 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:28.750922 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:28.752155 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.753205 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.754966 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.755447 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.755542 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:28.757052 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:28.757057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:28.757060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:28.757077 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:28.757106 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:28.757117 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:28.757182 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:28.757398 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 03:07:28.757475 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:07:28.759105 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:28.747135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:35097 (local address 127.5.122.2:57491)
0807 03:07:28.748168 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:28.748212 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:28.750501 (+ 2289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:28.751151 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:28.751162 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:28.751265 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:28.751987 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:28.752009 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.753373 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.753377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:28.754800 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:28.754813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:28.755668 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:28.755676 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:28.755986 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:28.756723 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:28.756727 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:28.756774 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:28.756877 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:28.757201 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:28.757211 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:28.757218 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 03:07:28.757392 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 03:07:28.757777 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":312,"threads_started":1}
I20260807 03:07:28.760396 3196 heartbeater.cc:347] Connected to a master server at 127.5.122.62:35097
I20260807 03:07:28.760675 3196 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:28.761389 3196 heartbeater.cc:511] Master 127.5.122.62:35097 requested a full tablet report, sending...
I20260807 03:07:28.762658 2745 ts_manager.cc:194] Registered new tserver with Master: 04f6488bcd934c62999c813f53040a07 (127.5.122.2:36935)
I20260807 03:07:28.763429 2745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.2:57491
W20260807 03:07:29.086979 3204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:29.087255 3204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:29.087352 3204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:29.094376 3204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:29.094466 3204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:29.094583 3204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.3
I20260807 03:07:29.103582 3204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:34657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.3
--webserver_port=0
--tserver_master_addrs=127.5.122.62:35097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.3
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:29.104796 3204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:29.106534 3204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:29.119025 3210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:29.119099 3212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:29.119068 3216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:29.119904 3204 server_base.cc:1034] running on GCE node
I20260807 03:07:29.120451 3204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:29.121331 3204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:29.122524 3204 hybrid_clock.cc:648] HybridClock initialized: now 1786072049122479 us; error 37 us; skew 500 ppm
I20260807 03:07:29.122805 3204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 03:07:29.124867 3204 webserver.cc:460] Webserver started at http://127.5.122.3:38245/ using document root <none> and password file <none>
I20260807 03:07:29.125505 3204 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:29.125604 3204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:29.125891 3204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:29.128557 3204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data/instance:
uuid: "35233d0e69974ba4a717bbd5a7e09ac8"
format_stamp: "Formatted at 2026-08-07 03:07:29 on dist-test-slave-fgb3"
server_key: "5097934dbe94700f44cf2f933257843f"
server_key_iv: "0a2cab067321610f0fdf07c6d6f14fae"
server_key_version: "encryptionkey@0"
I20260807 03:07:29.129313 3204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance:
uuid: "35233d0e69974ba4a717bbd5a7e09ac8"
format_stamp: "Formatted at 2026-08-07 03:07:29 on dist-test-slave-fgb3"
server_key: "5097934dbe94700f44cf2f933257843f"
server_key_iv: "0a2cab067321610f0fdf07c6d6f14fae"
server_key_version: "encryptionkey@0"
I20260807 03:07:29.133888 3204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 03:07:29.137053 3226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:29.138239 3204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:07:29.138427 3204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal
uuid: "35233d0e69974ba4a717bbd5a7e09ac8"
format_stamp: "Formatted at 2026-08-07 03:07:29 on dist-test-slave-fgb3"
server_key: "5097934dbe94700f44cf2f933257843f"
server_key_iv: "0a2cab067321610f0fdf07c6d6f14fae"
server_key_version: "encryptionkey@0"
I20260807 03:07:29.138622 3204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:29.168989 3204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:29.169896 3204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:29.170161 3204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:29.171263 3204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 03:07:29.172742 3204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 03:07:29.172842 3204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:29.172950 3204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 03:07:29.173025 3204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:29.202873 3204 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.3:39619
I20260807 03:07:29.202891 3382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.3:39619 every 8 connection(s)
I20260807 03:07:29.204216 3204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/data/info.pb
I20260807 03:07:29.206485 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 3204
I20260807 03:07:29.206671 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal/instance
I20260807 03:07:29.207113 5608 external_mini_cluster.cc:1428] Setting key 7abdb96794be5a256ee505b9187dae15
I20260807 03:07:29.215772 3011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.206607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.3:46207 (local address 127.5.122.62:35097)
0807 03:07:29.206813 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.206819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.208361 (+ 1542us) server_negotiation.cc:407] Connection header received
0807 03:07:29.210094 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.210098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.210166 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.210281 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:29.211274 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.212004 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.213423 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.213836 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.213919 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.215157 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:29.215163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:29.215166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:29.215184 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:29.215216 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:29.215229 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:29.215311 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.215465 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.215541 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:07:29.216704 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.207191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:35097 (local address 127.5.122.3:46207)
0807 03:07:29.208197 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.208242 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:29.209853 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:29.210483 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:29.210492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:29.210577 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:07:29.211121 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.211138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.212123 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.212127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.213318 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.213330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.214015 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.214020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.214210 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.214885 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 03:07:29.214887 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:07:29.214925 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 03:07:29.214998 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:07:29.215313 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:07:29.215319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:07:29.215324 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 03:07:29.215440 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 03:07:29.215710 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":316,"threads_started":1}
I20260807 03:07:29.217941 3383 heartbeater.cc:347] Connected to a master server at 127.5.122.62:35097
I20260807 03:07:29.218204 3383 heartbeater.cc:464] Registering TS with master...
I20260807 03:07:29.218964 3383 heartbeater.cc:511] Master 127.5.122.62:35097 requested a full tablet report, sending...
I20260807 03:07:29.220315 2745 ts_manager.cc:194] Registered new tserver with Master: 35233d0e69974ba4a717bbd5a7e09ac8 (127.5.122.3:39619)
I20260807 03:07:29.221184 2745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.3:46207
I20260807 03:07:29.221912 5608 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 03:07:29.231746 3011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.226207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.5.122.62:35097)
0807 03:07:29.226390 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.226394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.226814 (+ 420us) server_negotiation.cc:407] Connection header received
0807 03:07:29.226972 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.226976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.227048 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.227129 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:29.227615 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.228284 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.229623 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.230029 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.230131 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.231127 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:29.231132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:29.231135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:29.231152 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:29.231181 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:29.231193 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:29.231265 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.231390 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.231475 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 03:07:29.235332 2741 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 03:07:29.242000 2741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 03:07:29.263402 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.252896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43430)
0807 03:07:29.253847 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.253895 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:29.254075 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:29.255426 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:29.255435 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:29.255468 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:29.255925 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.255937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.258170 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.258176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.260529 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.260544 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.262515 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.262521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.262745 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.262807 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 03:07:29.262906 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 03:07:29.263086 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":1008,"threads_started":1}
I20260807 03:07:29.265071 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.253155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.5.122.1:40191)
0807 03:07:29.254770 (+ 1615us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.254792 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.254824 (+ 32us) server_negotiation.cc:407] Connection header received
0807 03:07:29.254932 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.254940 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.255190 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.256142 (+ 952us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:29.256364 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.258008 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.260791 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.262273 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.264238 (+ 1965us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.264472 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.264550 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.264713 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1478,"thread_start_us":1361,"threads_started":1}
I20260807 03:07:29.268748 2917 tablet_service.cc:1467] Processing CreateTablet for tablet 356ee915d9534352a143d84c57302dd3 (DEFAULT_TABLE table=test-table [id=bc811658c901457cbaec43eb3185c6a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:29.270332 2917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356ee915d9534352a143d84c57302dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:29.271090 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.253139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39619 (local address 127.0.0.1:60144)
0807 03:07:29.263633 (+ 10494us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.263651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:29.263743 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:29.265075 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:29.265080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:29.265110 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:29.265533 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.265546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.267382 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.267388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.269022 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.269033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.270348 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.270355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.270555 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.270587 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 03:07:29.270685 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 03:07:29.270821 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10407,"thread_start_us":12330,"threads_started":1}
I20260807 03:07:29.272145 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.253695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.5.122.3:39619)
0807 03:07:29.264242 (+ 10547us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.264261 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.264295 (+ 34us) server_negotiation.cc:407] Connection header received
0807 03:07:29.264390 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.264397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.264651 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.265343 (+ 692us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:29.265841 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.267236 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.269161 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.270142 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.270259 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.270555 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.271608 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.271766 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10371,"thread_start_us":10237,"threads_started":1}
I20260807 03:07:29.273206 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.253510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.2:36935 (local address 127.0.0.1:36626)
0807 03:07:29.264970 (+ 11460us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.264988 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:29.265366 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:29.266366 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:29.266370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:29.266394 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:29.266787 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.266799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.268732 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.268737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.270683 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.270698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.272528 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.272534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.272698 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.272728 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 03:07:29.272821 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 03:07:29.272938 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11263,"thread_start_us":11291,"threads_started":1}
I20260807 03:07:29.273363 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.254164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.5.122.2:36935)
0807 03:07:29.264506 (+ 10342us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.264514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.265078 (+ 564us) server_negotiation.cc:407] Connection header received
0807 03:07:29.265667 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.265675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.265876 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.266027 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:29.266982 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.268603 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.270886 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.272250 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.272384 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.272612 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.272897 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.273054 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10192,"thread_start_us":10078,"threads_started":1}
I20260807 03:07:29.275604 3286 tablet_service.cc:1467] Processing CreateTablet for tablet 356ee915d9534352a143d84c57302dd3 (DEFAULT_TABLE table=test-table [id=bc811658c901457cbaec43eb3185c6a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:29.276840 3286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356ee915d9534352a143d84c57302dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:29.277060 3107 tablet_service.cc:1467] Processing CreateTablet for tablet 356ee915d9534352a143d84c57302dd3 (DEFAULT_TABLE table=test-table [id=bc811658c901457cbaec43eb3185c6a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 03:07:29.278355 3107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356ee915d9534352a143d84c57302dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:29.285969 3402 tablet_bootstrap.cc:492] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: Bootstrap starting.
I20260807 03:07:29.289004 3404 tablet_bootstrap.cc:492] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Bootstrap starting.
I20260807 03:07:29.289670 3402 tablet_bootstrap.cc:654] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:29.290668 3402 log.cc:824] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:29.292258 3407 tablet_bootstrap.cc:492] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Bootstrap starting.
I20260807 03:07:29.293241 3404 tablet_bootstrap.cc:654] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:29.293622 3402 tablet_bootstrap.cc:492] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: No bootstrap required, opened a new log
I20260807 03:07:29.293895 3402 ts_tablet_manager.cc:1397] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 03:07:29.294183 3404 log.cc:824] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:29.295576 3407 tablet_bootstrap.cc:654] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:29.296571 3404 tablet_bootstrap.cc:492] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: No bootstrap required, opened a new log
I20260807 03:07:29.296686 3407 log.cc:824] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:29.296806 3404 ts_tablet_manager.cc:1397] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260807 03:07:29.299556 3407 tablet_bootstrap.cc:492] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: No bootstrap required, opened a new log
I20260807 03:07:29.299814 3407 ts_tablet_manager.cc:1397] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260807 03:07:29.303591 3402 raft_consensus.cc:348] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.304011 3402 raft_consensus.cc:374] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:29.304112 3402 raft_consensus.cc:729] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0602f36d77b4594914a48d7cb3d80ca, State: Initialized, Role: FOLLOWER
I20260807 03:07:29.304315 3404 raft_consensus.cc:348] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.304667 3404 raft_consensus.cc:374] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:29.304759 3404 raft_consensus.cc:729] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35233d0e69974ba4a717bbd5a7e09ac8, State: Initialized, Role: FOLLOWER
I20260807 03:07:29.304692 3402 consensus_queue.cc:260] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.305399 3404 consensus_queue.cc:260] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.305764 3009 heartbeater.cc:503] Master 127.5.122.62:35097 was elected leader, sending a full tablet report...
I20260807 03:07:29.306327 3402 ts_tablet_manager.cc:1428] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260807 03:07:29.309119 3407 raft_consensus.cc:348] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.309520 3407 raft_consensus.cc:374] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:29.309612 3407 raft_consensus.cc:729] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f6488bcd934c62999c813f53040a07, State: Initialized, Role: FOLLOWER
I20260807 03:07:29.310068 3407 consensus_queue.cc:260] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
W20260807 03:07:29.315477 3010 tablet.cc:2369] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:07:29.317443 3404 ts_tablet_manager.cc:1428] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Time spent starting tablet: real 0.021s user 0.011s sys 0.005s
I20260807 03:07:29.318126 3383 heartbeater.cc:503] Master 127.5.122.62:35097 was elected leader, sending a full tablet report...
I20260807 03:07:29.320587 3196 heartbeater.cc:503] Master 127.5.122.62:35097 was elected leader, sending a full tablet report...
I20260807 03:07:29.321281 3407 ts_tablet_manager.cc:1428] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Time spent starting tablet: real 0.021s user 0.015s sys 0.007s
I20260807 03:07:29.422200 3413 raft_consensus.cc:482] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 03:07:29.422503 3413 raft_consensus.cc:504] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.424211 3413 leader_election.cc:288] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0602f36d77b4594914a48d7cb3d80ca (127.5.122.1:40191), 35233d0e69974ba4a717bbd5a7e09ac8 (127.5.122.3:39619)
I20260807 03:07:29.432452 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.424726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:59543 (local address 127.5.122.3:39619)
0807 03:07:29.425128 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.425134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.425636 (+ 502us) server_negotiation.cc:407] Connection header received
0807 03:07:29.425757 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.425762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.425847 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.425926 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:29.426608 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.427646 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.430291 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.431479 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.431605 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.431732 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.432082 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.432205 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260807 03:07:29.432468 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.424698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.2:45427 (local address 127.5.122.1:40191)
0807 03:07:29.424969 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.424976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.425415 (+ 439us) server_negotiation.cc:407] Connection header received
0807 03:07:29.425598 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.425604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.425699 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.425801 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 03:07:29.426504 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.427547 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.430349 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.431493 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.431605 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.431706 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.432087 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.432204 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 03:07:29.432608 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.424488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.3:39619 (local address 127.5.122.2:59543)
0807 03:07:29.425537 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.425552 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:29.425650 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:29.425961 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:29.425965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:29.426001 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:29.426461 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.426473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.427764 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.427769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.430088 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.430103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.431671 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.431682 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.431911 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.431958 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 03:07:29.432050 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 03:07:29.432222 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"mutex_wait_us":187,"thread_start_us":745,"threads_started":1}
I20260807 03:07:29.432967 3414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.424423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.5.122.2:45427)
0807 03:07:29.425295 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.425320 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 03:07:29.425475 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:07:29.425819 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:07:29.425822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:07:29.425841 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 03:07:29.426367 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.426380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.427660 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.427664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.430088 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:07:29.430103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.431698 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.431706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:07:29.431905 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.431967 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 03:07:29.432045 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 03:07:29.432222 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":134,"spinlock_wait_cycles":896,"thread_start_us":483,"threads_started":1}
I20260807 03:07:29.433494 3314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356ee915d9534352a143d84c57302dd3" candidate_uuid: "04f6488bcd934c62999c813f53040a07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" is_pre_election: true
I20260807 03:07:29.433869 2940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356ee915d9534352a143d84c57302dd3" candidate_uuid: "04f6488bcd934c62999c813f53040a07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0602f36d77b4594914a48d7cb3d80ca" is_pre_election: true
I20260807 03:07:29.434098 3314 raft_consensus.cc:2454] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f6488bcd934c62999c813f53040a07 in term 0.
I20260807 03:07:29.434448 2940 raft_consensus.cc:2454] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f6488bcd934c62999c813f53040a07 in term 0.
I20260807 03:07:29.434819 3036 leader_election.cc:302] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f6488bcd934c62999c813f53040a07, 35233d0e69974ba4a717bbd5a7e09ac8; no voters:
I20260807 03:07:29.435225 3413 raft_consensus.cc:2781] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 03:07:29.435351 3413 raft_consensus.cc:482] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 03:07:29.435446 3413 raft_consensus.cc:3037] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:29.437592 3413 raft_consensus.cc:504] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.438162 3413 leader_election.cc:288] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [CANDIDATE]: Term 1 election: Requested vote from peers b0602f36d77b4594914a48d7cb3d80ca (127.5.122.1:40191), 35233d0e69974ba4a717bbd5a7e09ac8 (127.5.122.3:39619)
I20260807 03:07:29.438544 2940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356ee915d9534352a143d84c57302dd3" candidate_uuid: "04f6488bcd934c62999c813f53040a07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0602f36d77b4594914a48d7cb3d80ca"
I20260807 03:07:29.438627 3314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356ee915d9534352a143d84c57302dd3" candidate_uuid: "04f6488bcd934c62999c813f53040a07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35233d0e69974ba4a717bbd5a7e09ac8"
I20260807 03:07:29.438706 2940 raft_consensus.cc:3037] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:29.438828 3314 raft_consensus.cc:3037] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:29.440781 2940 raft_consensus.cc:2454] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f6488bcd934c62999c813f53040a07 in term 1.
I20260807 03:07:29.440914 3314 raft_consensus.cc:2454] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f6488bcd934c62999c813f53040a07 in term 1.
I20260807 03:07:29.441116 3040 leader_election.cc:302] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f6488bcd934c62999c813f53040a07, b0602f36d77b4594914a48d7cb3d80ca; no voters:
I20260807 03:07:29.441452 3413 raft_consensus.cc:2781] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:29.441857 3413 raft_consensus.cc:686] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 1 LEADER]: Becoming Leader. State: Replica: 04f6488bcd934c62999c813f53040a07, State: Running, Role: LEADER
I20260807 03:07:29.442287 3413 consensus_queue.cc:237] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } }
I20260807 03:07:29.446547 2741 catalog_manager.cc:5495] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04f6488bcd934c62999c813f53040a07 (127.5.122.2). New cstate: current_term: 1 leader_uuid: "04f6488bcd934c62999c813f53040a07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04f6488bcd934c62999c813f53040a07" member_type: VOTER last_known_addr { host: "127.5.122.2" port: 36935 } health_report { overall_health: HEALTHY } } }
W20260807 03:07:29.455745 3384 tablet.cc:2369] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:07:29.495945 3202 tablet.cc:2369] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:07:29.506407 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.501482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.5.122.2:36935)
0807 03:07:29.501692 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.501696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.501713 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:07:29.501886 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.501890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.502013 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.502134 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:07:29.502449 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.503199 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.504576 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.504981 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.505069 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.505454 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:07:29.505920 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:07:29.506031 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.506066 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.506127 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 03:07:29.514184 2940 raft_consensus.cc:1261] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 1 FOLLOWER]: Refusing update from remote peer 04f6488bcd934c62999c813f53040a07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:29.514254 3314 raft_consensus.cc:1261] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 1 FOLLOWER]: Refusing update from remote peer 04f6488bcd934c62999c813f53040a07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:07:29.515172 3413 consensus_queue.cc:1035] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0602f36d77b4594914a48d7cb3d80ca" member_type: VOTER last_known_addr { host: "127.5.122.1" port: 40191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:29.515446 3417 consensus_queue.cc:1035] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35233d0e69974ba4a717bbd5a7e09ac8" member_type: VOTER last_known_addr { host: "127.5.122.3" port: 39619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:07:29.526011 3424 mvcc.cc:205] Tried to move back new op lower bound from 7315751114800271360 to 7315751114519482368. Current Snapshot: MvccSnapshot[applied={T|T < 7315751114800271360}]
I20260807 03:07:29.527403 3425 mvcc.cc:205] Tried to move back new op lower bound from 7315751114800271360 to 7315751114519482368. Current Snapshot: MvccSnapshot[applied={T|T < 7315751114800271360}]
I20260807 03:07:29.527412 3426 mvcc.cc:205] Tried to move back new op lower bound from 7315751114800271360 to 7315751114519482368. Current Snapshot: MvccSnapshot[applied={T|T < 7315751114800271360}]
I20260807 03:07:29.539740 2741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:29.540040 2741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 03:07:29.543597 2741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8de297c13af64b99a4304a67ad9fe62b: Sending DeleteTablet for 3 replicas of tablet 356ee915d9534352a143d84c57302dd3
I20260807 03:07:29.544636 2917 tablet_service.cc:1514] Processing DeleteTablet for tablet 356ee915d9534352a143d84c57302dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:29 UTC) from {username='slave'} at 127.0.0.1:43430
I20260807 03:07:29.544715 3286 tablet_service.cc:1514] Processing DeleteTablet for tablet 356ee915d9534352a143d84c57302dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:29 UTC) from {username='slave'} at 127.0.0.1:60144
I20260807 03:07:29.544893 3107 tablet_service.cc:1514] Processing DeleteTablet for tablet 356ee915d9534352a143d84c57302dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 03:07:29 UTC) from {username='slave'} at 127.0.0.1:36626
I20260807 03:07:29.545754 3445 tablet_replica.cc:330] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: stopping tablet replica
I20260807 03:07:29.546243 3445 raft_consensus.cc:2229] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 1 LEADER]: Raft consensus shutting down.
I20260807 03:07:29.546397 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 2824
I20260807 03:07:29.547072 3445 raft_consensus.cc:2258] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:29.556249 3442 tablet_replica.cc:330] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: stopping tablet replica
I20260807 03:07:29.556712 3442 raft_consensus.cc:2229] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:29.557260 3442 raft_consensus.cc:2258] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:29.559567 3442 ts_tablet_manager.cc:1905] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:29.559574 3445 ts_tablet_manager.cc:1905] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 03:07:29.563515 3441 tablet_replica.cc:330] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca: stopping tablet replica
I20260807 03:07:29.565726 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.547829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43444 (local address 127.5.122.1:40191)
0807 03:07:29.548017 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.548022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.557123 (+ 9101us) server_negotiation.cc:407] Connection header received
0807 03:07:29.557582 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.557589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.557743 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.557852 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:29.558590 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.559747 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.561648 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.562339 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.562435 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.563912 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:29.563924 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:29.563941 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:29.564677 (+ 736us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:29.564736 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:29.564761 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:29.564865 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.565225 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.565336 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 03:07:29.566749 3445 ts_tablet_manager.cc:1918] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:29.566895 3003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:29.566947 3445 log.cc:1196] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-1/wal/wals/356ee915d9534352a143d84c57302dd3
I20260807 03:07:29.567117 3442 ts_tablet_manager.cc:1918] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 03:07:29.567260 3442 log.cc:1196] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Deleting WAL directory at /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786071733698599-5608-0/minicluster-data/ts-2/wal/wals/356ee915d9534352a143d84c57302dd3
I20260807 03:07:29.567530 3445 ts_tablet_manager.cc:1939] T 356ee915d9534352a143d84c57302dd3 P 04f6488bcd934c62999c813f53040a07: Deleting consensus metadata
I20260807 03:07:29.567749 3442 ts_tablet_manager.cc:1939] T 356ee915d9534352a143d84c57302dd3 P 35233d0e69974ba4a717bbd5a7e09ac8: Deleting consensus metadata
I20260807 03:07:29.569339 2725 catalog_manager.cc:4841] TS 35233d0e69974ba4a717bbd5a7e09ac8 (127.5.122.3:39619): tablet 356ee915d9534352a143d84c57302dd3 (table test-table [id=bc811658c901457cbaec43eb3185c6a5]) successfully deleted
I20260807 03:07:29.569473 2723 catalog_manager.cc:4841] TS 04f6488bcd934c62999c813f53040a07 (127.5.122.2:36935): tablet 356ee915d9534352a143d84c57302dd3 (table test-table [id=bc811658c901457cbaec43eb3185c6a5]) successfully deleted
I20260807 03:07:29.569849 3441 raft_consensus.cc:2229] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 03:07:29.656406 3441 raft_consensus.cc:2258] T 356ee915d9534352a143d84c57302dd3 P b0602f36d77b4594914a48d7cb3d80ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 03:07:29.656545 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 2824
W20260807 03:07:29.679044 2727 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.1:40191 (error 108)
I20260807 03:07:29.679200 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3014
I20260807 03:07:29.680920 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.680471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43454)
0807 03:07:29.680675 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.680787 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 03:07:29.681603 2727 catalog_manager.cc:4584] TS b0602f36d77b4594914a48d7cb3d80ca (127.5.122.1:40191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 356ee915d9534352a143d84c57302dd3: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
I20260807 03:07:29.692019 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.680601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.5.122.2:36935)
0807 03:07:29.680792 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.680798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.680817 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:07:29.680880 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.680886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.680989 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.681088 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:29.681546 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.682782 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.684014 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.684698 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.684838 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.690432 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:29.690443 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:29.690462 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:29.691250 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:29.691312 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:29.691337 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:29.691442 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.691526 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.691628 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 03:07:29.692729 3189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:29.735210 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.734789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43470)
0807 03:07:29.734968 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.735057 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 03:07:29.736238 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.735850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43480)
0807 03:07:29.736007 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.736093 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 03:07:29.796793 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.796280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43494)
0807 03:07:29.796503 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.796598 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 03:07:29.797827 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.797473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43504)
0807 03:07:29.797618 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.797694 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 03:07:29.804142 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 3014
I20260807 03:07:29.826176 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3204
I20260807 03:07:29.832556 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.826923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.5.122.3:39619)
0807 03:07:29.827095 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:29.827100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:29.827120 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:07:29.827290 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:29.827294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:29.827372 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:29.827480 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:29.827878 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.828579 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.829848 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:29.830229 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:29.830357 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:29.831293 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:29.831301 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:29.831315 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:29.831861 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:29.831908 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:29.831927 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:29.832035 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:29.832154 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 03:07:29.832258 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 03:07:29.833122 3375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:29.878369 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.877839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43506)
0807 03:07:29.878068 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.878156 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 03:07:29.879354 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:29.878974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.1:40191 (local address 127.0.0.1:43514)
0807 03:07:29.879172 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 03:07:29.879247 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.1:40191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 03:07:29.919252 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 3204
I20260807 03:07:29.940312 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 2699
I20260807 03:07:29.941404 2786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:30.047798 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 2699
2026-08-07T03:07:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2819 ms)
[----------] 6 tests from EncryptionPolicyTest (21316 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:30 dist-test-slave-fgb3 krb5kdc[3456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:30 dist-test-slave-fgb3 krb5kdc[3456](info): set up 2 sockets
Aug 07 03:07:30 dist-test-slave-fgb3 krb5kdc[3456](info): commencing operation
krb5kdc: starting...
W20260807 03:07:31.245740 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:31 dist-test-slave-fgb3 krb5kdc[3456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:31.542042 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40985
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:31.868727 3473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:31.869024 3473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:31.869118 3473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:31.876477 3473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:31.876601 3473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:31.876653 3473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:31.876722 3473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:31.876765 3473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:31.886960 3473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40985
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:31.888374 3473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:31.889638 3473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:31.901019 3478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:31.901099 3479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:31.901059 3481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:31.902215 3473 server_base.cc:1034] running on GCE node
I20260807 03:07:31.902997 3473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:31.904356 3473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:31.905570 3473 hybrid_clock.cc:648] HybridClock initialized: now 1786072051905511 us; error 46 us; skew 500 ppm
I20260807 03:07:31.905912 3473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:31 dist-test-slave-fgb3 krb5kdc[3456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072051, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:31.908936 3473 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:31.910514 3473 webserver.cc:460] Webserver started at http://127.0.0.1:36299/ using document root <none> and password file <none>
I20260807 03:07:31.911206 3473 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:31.911324 3473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:31.911617 3473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:31.914758 3473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "e768164533cb411aa3e7c3d638832914"
format_stamp: "Formatted at 2026-08-07 03:07:31 on dist-test-slave-fgb3"
server_key: "d1652b65851a21d429b276496c71983b"
server_key_iv: "3d65c614cfdb8c9446cf6d1a5723e227"
server_key_version: "encryptionkey@0"
I20260807 03:07:31.915530 3473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "e768164533cb411aa3e7c3d638832914"
format_stamp: "Formatted at 2026-08-07 03:07:31 on dist-test-slave-fgb3"
server_key: "d1652b65851a21d429b276496c71983b"
server_key_iv: "3d65c614cfdb8c9446cf6d1a5723e227"
server_key_version: "encryptionkey@0"
I20260807 03:07:31.920785 3473 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 03:07:31.924284 3491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:31.925637 3473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:07:31.925815 3473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "e768164533cb411aa3e7c3d638832914"
format_stamp: "Formatted at 2026-08-07 03:07:31 on dist-test-slave-fgb3"
server_key: "d1652b65851a21d429b276496c71983b"
server_key_iv: "3d65c614cfdb8c9446cf6d1a5723e227"
server_key_version: "encryptionkey@0"
I20260807 03:07:31.926010 3473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:31.957243 3473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:31.961318 3473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:31.961603 3473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:31.979667 3473 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40985
I20260807 03:07:31.979722 3564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40985 every 8 connection(s)
I20260807 03:07:31.981055 3473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:31.984850 3565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:31.989643 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 3473
I20260807 03:07:31.989881 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:31.990756 5608 external_mini_cluster.cc:1428] Setting key fb4f014faf300bfe03985c63465bb211
I20260807 03:07:31.996451 3565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914: Bootstrap starting.
I20260807 03:07:32.000030 3565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:32.001241 3565 log.cc:824] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:32.004088 3565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914: No bootstrap required, opened a new log
Aug 07 03:07:32 dist-test-slave-fgb3 krb5kdc[3456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 03:07:32.012688 3565 raft_consensus.cc:348] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768164533cb411aa3e7c3d638832914" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40985 } }
I20260807 03:07:32.013118 3565 raft_consensus.cc:374] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:32.013242 3565 raft_consensus.cc:729] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e768164533cb411aa3e7c3d638832914, State: Initialized, Role: FOLLOWER
I20260807 03:07:32.013931 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:31.992707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42357 (local address 127.0.0.1:40985)
0807 03:07:31.993487 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:31.993501 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:31.993545 (+ 44us) server_negotiation.cc:407] Connection header received
0807 03:07:31.995330 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:31.995353 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:31.995732 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:31.996048 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:31.997809 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:31.999189 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:32.000953 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:32.001644 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:32.001872 (+ 228us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:32.006050 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:32.006058 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:32.006076 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:32.006109 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:32.009289 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:32.010099 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:32.010106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:32.010113 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:32.010285 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:32.010834 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:32.010839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:32.010843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:32.011586 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:32.011907 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:32.012412 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 03:07:32.012755 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":327,"threads_started":1}
I20260807 03:07:32.013870 3565 consensus_queue.cc:260] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768164533cb411aa3e7c3d638832914" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40985 } }
I20260807 03:07:32.014118 3565 raft_consensus.cc:388] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:32.014253 3565 raft_consensus.cc:482] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:32.014442 3565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:32.016916 3565 raft_consensus.cc:504] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768164533cb411aa3e7c3d638832914" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40985 } }
I20260807 03:07:32.017490 3565 leader_election.cc:302] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e768164533cb411aa3e7c3d638832914; no voters:
I20260807 03:07:32.017872 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:32.018116 3565 leader_election.cc:288] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:32.018247 3572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:32.019997 3565 sys_catalog.cc:564] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:32.027179 3572 raft_consensus.cc:686] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [term 1 LEADER]: Becoming Leader. State: Replica: e768164533cb411aa3e7c3d638832914, State: Running, Role: LEADER
I20260807 03:07:32.027940 3572 consensus_queue.cc:237] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768164533cb411aa3e7c3d638832914" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40985 } }
I20260807 03:07:32.031502 3575 sys_catalog.cc:455] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e768164533cb411aa3e7c3d638832914. Latest consensus state: current_term: 1 leader_uuid: "e768164533cb411aa3e7c3d638832914" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768164533cb411aa3e7c3d638832914" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40985 } } }
I20260807 03:07:32.032562 3575 sys_catalog.cc:458] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:32.032090 3574 sys_catalog.cc:455] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e768164533cb411aa3e7c3d638832914" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768164533cb411aa3e7c3d638832914" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40985 } } }
I20260807 03:07:32.033017 3574 sys_catalog.cc:458] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:32.037146 3594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:32.044386 3594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:32.049752 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:32.029974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48091 (local address 127.0.0.1:40985)
0807 03:07:32.030244 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:32.030250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:32.031271 (+ 1021us) server_negotiation.cc:407] Connection header received
0807 03:07:32.031569 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:32.031574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:32.031658 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:32.031736 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:32.033797 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:32.034946 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:32.036166 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:32.036826 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:32.036968 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:32.040312 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:32.040319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:32.040325 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:32.040367 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:32.044218 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:32.047126 (+ 2908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:32.047131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:32.047135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:32.047287 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:32.047879 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:32.047884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:32.047887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:32.048286 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:32.048538 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:32.049228 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 03:07:32.049408 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260807 03:07:32.056617 3594 catalog_manager.cc:1347] Generated new cluster ID: d6a1f897e4a049dc9d42586d27025ce7
I20260807 03:07:32.056727 3594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:32.068833 3594 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:32.070667 3594 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:32.079356 3594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e768164533cb411aa3e7c3d638832914: Generated new TSK 0
I20260807 03:07:32.080258 3594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:32.089638 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3473
I20260807 03:07:32.090965 3557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:32.179232 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 3473
W20260807 03:07:32.199086 3454 connection.cc:535] client connection to 127.0.0.1:40985 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40985): Connection reset by peer (error 104)
2026-08-07T03:07:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2124 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:32 dist-test-slave-fgb3 krb5kdc[3610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:32 dist-test-slave-fgb3 krb5kdc[3610](info): set up 2 sockets
Aug 07 03:07:32 dist-test-slave-fgb3 krb5kdc[3610](info): commencing operation
krb5kdc: starting...
W20260807 03:07:33.392454 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:33 dist-test-slave-fgb3 krb5kdc[3610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:33.685169 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:34.017330 3632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:34.017635 3632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:34.017792 3632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:34.025398 3632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:34.025534 3632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:34.025631 3632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:34.025722 3632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:34.025792 3632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:34.035719 3632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42859
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:34.037086 3632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:34.038486 3632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:34.050698 3640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:34.050778 3644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:34.050730 3639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:34.051640 3632 server_base.cc:1034] running on GCE node
I20260807 03:07:34.052501 3632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:34.060001 3632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:34.061244 3632 hybrid_clock.cc:648] HybridClock initialized: now 1786072054061180 us; error 51 us; skew 500 ppm
I20260807 03:07:34.061685 3632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:34 dist-test-slave-fgb3 krb5kdc[3610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:34.064900 3632 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:34.066305 3632 webserver.cc:460] Webserver started at http://127.0.0.1:36963/ using document root <none> and password file <none>
I20260807 03:07:34.067018 3632 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:34.067124 3632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:34.067376 3632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:34.070238 3632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "5b5b3e472c2a4b1493b9776fe639bf49"
format_stamp: "Formatted at 2026-08-07 03:07:34 on dist-test-slave-fgb3"
server_key: "2c874da1d3d1df241d205c6e62c7d73e"
server_key_iv: "0d9f64196247f3719b9449efdac4c330"
server_key_version: "encryptionkey@0"
I20260807 03:07:34.071004 3632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "5b5b3e472c2a4b1493b9776fe639bf49"
format_stamp: "Formatted at 2026-08-07 03:07:34 on dist-test-slave-fgb3"
server_key: "2c874da1d3d1df241d205c6e62c7d73e"
server_key_iv: "0d9f64196247f3719b9449efdac4c330"
server_key_version: "encryptionkey@0"
I20260807 03:07:34.075879 3632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:07:34.079191 3654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:34.080418 3632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:07:34.080565 3632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "5b5b3e472c2a4b1493b9776fe639bf49"
format_stamp: "Formatted at 2026-08-07 03:07:34 on dist-test-slave-fgb3"
server_key: "2c874da1d3d1df241d205c6e62c7d73e"
server_key_iv: "0d9f64196247f3719b9449efdac4c330"
server_key_version: "encryptionkey@0"
I20260807 03:07:34.080719 3632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:34.107743 3632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:34.111898 3632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:34.112139 3632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:34.129920 3632 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42859
I20260807 03:07:34.129988 3735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42859 every 8 connection(s)
I20260807 03:07:34.131246 3632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:34.134900 3736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:34.138198 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 3632
I20260807 03:07:34.138433 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:34.138950 5608 external_mini_cluster.cc:1428] Setting key 06ad678bf9fbf50e370a764448edfd14
I20260807 03:07:34.146540 3736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49: Bootstrap starting.
I20260807 03:07:34.149883 3736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:34.150981 3736 log.cc:824] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:34.151400 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:34.140617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40533 (local address 127.0.0.1:42859)
0807 03:07:34.141303 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:34.141314 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:34.141355 (+ 41us) server_negotiation.cc:407] Connection header received
0807 03:07:34.142986 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:34.143006 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:34.143388 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:34.143696 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:34.144219 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:34.145560 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:34.147338 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:34.147990 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:34.148119 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:34.149283 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:34.149291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:34.149303 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:34.149794 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:34.149863 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:34.149889 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:34.150040 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:34.150340 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 03:07:34.150479 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":308,"threads_started":1}
I20260807 03:07:34.153792 3736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49: No bootstrap required, opened a new log
I20260807 03:07:34.154968 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:34.161310 3736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42859 } }
I20260807 03:07:34.161612 3736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:34.161700 3736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b5b3e472c2a4b1493b9776fe639bf49, State: Initialized, Role: FOLLOWER
I20260807 03:07:34.162209 3736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42859 } }
I20260807 03:07:34.162463 3736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:34.162606 3736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:34.162763 3736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:34.164774 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:34.159047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58447 (local address 127.0.0.1:42859)
0807 03:07:34.159176 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:34.159180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:34.159621 (+ 441us) server_negotiation.cc:407] Connection header received
0807 03:07:34.159776 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:34.159780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:34.159837 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:34.159917 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:34.160428 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:34.161130 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:34.162518 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:34.163034 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:34.163151 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:34.164128 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:34.164133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:34.164137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:34.164153 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:34.164186 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:34.164200 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:34.164286 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:34.164473 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 03:07:34.164518 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 03:07:34.164937 3736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42859 } }
I20260807 03:07:34.165481 3736 leader_election.cc:302] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b5b3e472c2a4b1493b9776fe639bf49; no voters:
I20260807 03:07:34.166093 3736 leader_election.cc:288] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:34.166178 3750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:34.166817 3750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [term 1 LEADER]: Becoming Leader. State: Replica: 5b5b3e472c2a4b1493b9776fe639bf49, State: Running, Role: LEADER
I20260807 03:07:34.167313 3750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42859 } }
I20260807 03:07:34.168026 3736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:34.169780 3756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b5b3e472c2a4b1493b9776fe639bf49. Latest consensus state: current_term: 1 leader_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42859 } } }
I20260807 03:07:34.170176 3756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:34.170725 3754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5b3e472c2a4b1493b9776fe639bf49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42859 } } }
I20260807 03:07:34.170996 3754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:34.174767 3766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:34.179242 3766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:34.187188 3766 catalog_manager.cc:1347] Generated new cluster ID: d5de98db098246d89a5cb46913f66247
I20260807 03:07:34.187292 3766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:34.195051 3766 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:34.196652 3766 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:34.207029 3766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b5b3e472c2a4b1493b9776fe639bf49: Generated new TSK 0
I20260807 03:07:34.208040 3766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:34.221568 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3632
W20260807 03:07:34.222662 3728 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40533
W20260807 03:07:34.223328 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3632: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:34.223495 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 3632
W20260807 03:07:34.244097 3607 connection.cc:535] client connection to 127.0.0.1:42859 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42859): Connection reset by peer (error 104)
2026-08-07T03:07:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:34 dist-test-slave-fgb3 krb5kdc[3783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:34 dist-test-slave-fgb3 krb5kdc[3783](info): set up 2 sockets
Aug 07 03:07:34 dist-test-slave-fgb3 krb5kdc[3783](info): commencing operation
krb5kdc: starting...
W20260807 03:07:35.439376 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:35 dist-test-slave-fgb3 krb5kdc[3783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:35.737336 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:36.068719 3798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:36.068984 3798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:36.069085 3798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:36.075992 3798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:36.076095 3798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:36.076146 3798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:36.076206 3798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:36.076248 3798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:36.085819 3798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:36.087234 3798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:36.088595 3798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:36.099723 3806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:36.099838 3809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:36.100529 3805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:36.100837 3798 server_base.cc:1034] running on GCE node
I20260807 03:07:36.101604 3798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:36.102991 3798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:36.104205 3798 hybrid_clock.cc:648] HybridClock initialized: now 1786072056104161 us; error 36 us; skew 500 ppm
I20260807 03:07:36.104560 3798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:36 dist-test-slave-fgb3 krb5kdc[3783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072056, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:36.111099 3798 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:36.112672 3798 webserver.cc:460] Webserver started at http://127.0.0.1:44699/ using document root <none> and password file <none>
I20260807 03:07:36.113381 3798 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:36.113511 3798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:36.113828 3798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:36.116950 3798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "4ba76daa50b742d9b3c05ac281f56a47"
format_stamp: "Formatted at 2026-08-07 03:07:36 on dist-test-slave-fgb3"
server_key: "ddaba099d269ea4d093d8f5456fb19e6"
server_key_iv: "3cf5f80e51d85950b92e479e46fe5643"
server_key_version: "encryptionkey@0"
I20260807 03:07:36.117722 3798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "4ba76daa50b742d9b3c05ac281f56a47"
format_stamp: "Formatted at 2026-08-07 03:07:36 on dist-test-slave-fgb3"
server_key: "ddaba099d269ea4d093d8f5456fb19e6"
server_key_iv: "3cf5f80e51d85950b92e479e46fe5643"
server_key_version: "encryptionkey@0"
I20260807 03:07:36.123188 3798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 03:07:36.126540 3819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:36.127761 3798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 03:07:36.127929 3798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "4ba76daa50b742d9b3c05ac281f56a47"
format_stamp: "Formatted at 2026-08-07 03:07:36 on dist-test-slave-fgb3"
server_key: "ddaba099d269ea4d093d8f5456fb19e6"
server_key_iv: "3cf5f80e51d85950b92e479e46fe5643"
server_key_version: "encryptionkey@0"
I20260807 03:07:36.128116 3798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:36.133764 3798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:36.137557 3798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:36.137850 3798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:36.155309 3798 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41873
I20260807 03:07:36.155351 3898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41873 every 8 connection(s)
I20260807 03:07:36.156603 3798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:36.160068 3900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:36.164973 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 3798
I20260807 03:07:36.165141 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:36.165647 5608 external_mini_cluster.cc:1428] Setting key f7818ab3f843c0672317a57e7cd133cc
I20260807 03:07:36.170559 3900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47: Bootstrap starting.
I20260807 03:07:36.172705 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:36.167323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57901 (local address 127.0.0.1:41873)
0807 03:07:36.168070 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:36.168080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:36.168124 (+ 44us) server_negotiation.cc:407] Connection header received
0807 03:07:36.169751 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:36.169772 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:36.170061 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:36.170409 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:36.171346 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:36.171351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:36.171363 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:36.171376 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:36.171432 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:36.171449 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:36.171602 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:36.171693 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 03:07:36.171842 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":345,"threads_started":1}
I20260807 03:07:36.173691 3900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:36.174714 3900 log.cc:824] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:36.175900 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:36.177177 3900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47: No bootstrap required, opened a new log
I20260807 03:07:36.182139 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:36.179962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57387 (local address 127.0.0.1:41873)
0807 03:07:36.180091 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:36.180095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:36.180456 (+ 361us) server_negotiation.cc:407] Connection header received
0807 03:07:36.180666 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:36.180670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:36.180710 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:36.180781 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:36.181616 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:36.181620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:36.181623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:36.181636 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:36.181671 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:36.181685 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:36.181764 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:36.181852 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 03:07:36.181892 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 03:07:36.184957 3900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba76daa50b742d9b3c05ac281f56a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41873 } }
I20260807 03:07:36.185276 3900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:36.185369 3900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba76daa50b742d9b3c05ac281f56a47, State: Initialized, Role: FOLLOWER
I20260807 03:07:36.185905 3900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba76daa50b742d9b3c05ac281f56a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41873 } }
I20260807 03:07:36.186122 3900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:36.186254 3900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:36.186448 3900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:36.188436 3900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba76daa50b742d9b3c05ac281f56a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41873 } }
I20260807 03:07:36.188870 3900 leader_election.cc:302] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ba76daa50b742d9b3c05ac281f56a47; no voters:
I20260807 03:07:36.189447 3900 leader_election.cc:288] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:36.189558 3913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:36.190166 3913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [term 1 LEADER]: Becoming Leader. State: Replica: 4ba76daa50b742d9b3c05ac281f56a47, State: Running, Role: LEADER
I20260807 03:07:36.190726 3913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba76daa50b742d9b3c05ac281f56a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41873 } }
I20260807 03:07:36.191273 3900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:36.193300 3914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ba76daa50b742d9b3c05ac281f56a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba76daa50b742d9b3c05ac281f56a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41873 } } }
I20260807 03:07:36.193689 3914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:36.195210 3915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ba76daa50b742d9b3c05ac281f56a47. Latest consensus state: current_term: 1 leader_uuid: "4ba76daa50b742d9b3c05ac281f56a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba76daa50b742d9b3c05ac281f56a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41873 } } }
I20260807 03:07:36.195451 3915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:36.195986 3923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:36.200243 3923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:36.208501 3923 catalog_manager.cc:1347] Generated new cluster ID: b0240debfdd64efbadc6eb96c15a0988
I20260807 03:07:36.208619 3923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:36.236071 3923 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:36.236493 3923 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:36.244113 3923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ba76daa50b742d9b3c05ac281f56a47: Generated new TSK 0
I20260807 03:07:36.244861 3923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:36.253458 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3798
W20260807 03:07:36.254576 3892 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57901
W20260807 03:07:36.255299 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3798: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:36.255452 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 3798
2026-08-07T03:07:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2035 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:36 dist-test-slave-fgb3 krb5kdc[3941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:36 dist-test-slave-fgb3 krb5kdc[3941](info): set up 2 sockets
Aug 07 03:07:36 dist-test-slave-fgb3 krb5kdc[3941](info): commencing operation
krb5kdc: starting...
W20260807 03:07:37.458874 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.004s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:37 dist-test-slave-fgb3 krb5kdc[3941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:37.734983 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:38.050510 3961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:38.050840 3961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:38.050987 3961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:38.058234 3961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:38.058382 3961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:38.058475 3961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:38.058560 3961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:38.058624 3961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:38.069571 3961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:38.071008 3961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:38.072351 3961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:38.083900 3971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:38.084024 3968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:38.084059 3967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:38.084703 3961 server_base.cc:1034] running on GCE node
I20260807 03:07:38.085400 3961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:38.086862 3961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:38.088060 3961 hybrid_clock.cc:648] HybridClock initialized: now 1786072058088016 us; error 37 us; skew 500 ppm
I20260807 03:07:38.088389 3961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:38 dist-test-slave-fgb3 krb5kdc[3941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072058, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:38.091507 3961 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:38.092968 3961 webserver.cc:460] Webserver started at http://127.0.0.1:35777/ using document root <none> and password file <none>
I20260807 03:07:38.093631 3961 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:38.093746 3961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:38.094030 3961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:38.097033 3961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "94e5b6d88d3a4a80b7162ce83a599300"
format_stamp: "Formatted at 2026-08-07 03:07:38 on dist-test-slave-fgb3"
server_key: "0296e5706544b8785eb0ebe1badb6b58"
server_key_iv: "d6cee4b572fa8fdd3cf192067bd5febc"
server_key_version: "encryptionkey@0"
I20260807 03:07:38.097749 3961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "94e5b6d88d3a4a80b7162ce83a599300"
format_stamp: "Formatted at 2026-08-07 03:07:38 on dist-test-slave-fgb3"
server_key: "0296e5706544b8785eb0ebe1badb6b58"
server_key_iv: "d6cee4b572fa8fdd3cf192067bd5febc"
server_key_version: "encryptionkey@0"
I20260807 03:07:38.102658 3961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260807 03:07:38.105804 3981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:38.107080 3961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:07:38.107239 3961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "94e5b6d88d3a4a80b7162ce83a599300"
format_stamp: "Formatted at 2026-08-07 03:07:38 on dist-test-slave-fgb3"
server_key: "0296e5706544b8785eb0ebe1badb6b58"
server_key_iv: "d6cee4b572fa8fdd3cf192067bd5febc"
server_key_version: "encryptionkey@0"
I20260807 03:07:38.107406 3961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:38.133132 3961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:38.137007 3961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:38.137265 3961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:38.154083 3961 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38185
I20260807 03:07:38.154147 4054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38185 every 8 connection(s)
I20260807 03:07:38.155409 3961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:38.157564 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 3961
I20260807 03:07:38.157747 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:38.158352 5608 external_mini_cluster.cc:1428] Setting key 28bccf5a4f6e9252749ac1cb90f14172
I20260807 03:07:38.159556 4055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 03:07:38 dist-test-slave-fgb3 krb5kdc[3941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 03:07:38.172925 4055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300: Bootstrap starting.
I20260807 03:07:38.175997 4055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:38.177186 4055 log.cc:824] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:38.179919 4055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300: No bootstrap required, opened a new log
I20260807 03:07:38.180212 4059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:38.160082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44277 (local address 127.0.0.1:38185)
0807 03:07:38.160823 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:38.160833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:38.160873 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:07:38.162436 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:38.162463 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:38.162831 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:38.163143 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:38.164808 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:38.166109 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:38.167849 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:38.168477 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:38.168604 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:38.173404 (+ 4800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:38.173412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:38.173425 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:38.173471 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:38.176353 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:38.177097 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:38.177103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:38.177108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:38.177270 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:38.177644 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:38.177649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:38.177652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:38.178167 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:38.178503 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:38.178900 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 03:07:38.179289 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":338,"threads_started":1}
I20260807 03:07:38.183558 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:38.187675 4055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e5b6d88d3a4a80b7162ce83a599300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260807 03:07:38.188156 4055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:38.188263 4055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e5b6d88d3a4a80b7162ce83a599300, State: Initialized, Role: FOLLOWER
I20260807 03:07:38.188844 4055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e5b6d88d3a4a80b7162ce83a599300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260807 03:07:38.189050 4055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:38.189177 4055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:38.189322 4055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:38.191511 4055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e5b6d88d3a4a80b7162ce83a599300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260807 03:07:38.191970 4055 leader_election.cc:302] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94e5b6d88d3a4a80b7162ce83a599300; no voters:
I20260807 03:07:38.192553 4055 leader_election.cc:288] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:38.192662 4070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:38.194557 4070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [term 1 LEADER]: Becoming Leader. State: Replica: 94e5b6d88d3a4a80b7162ce83a599300, State: Running, Role: LEADER
I20260807 03:07:38.195272 4055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:38.195165 4070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e5b6d88d3a4a80b7162ce83a599300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260807 03:07:38.199854 4074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94e5b6d88d3a4a80b7162ce83a599300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e5b6d88d3a4a80b7162ce83a599300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } } }
I20260807 03:07:38.199878 4071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94e5b6d88d3a4a80b7162ce83a599300. Latest consensus state: current_term: 1 leader_uuid: "94e5b6d88d3a4a80b7162ce83a599300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e5b6d88d3a4a80b7162ce83a599300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } } }
I20260807 03:07:38.200305 4074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:38.200306 4071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:38.201135 4079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:38.202996 4059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:38.187767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43975 (local address 127.0.0.1:38185)
0807 03:07:38.187946 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:38.187951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:38.188387 (+ 436us) server_negotiation.cc:407] Connection header received
0807 03:07:38.188534 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:38.188539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:38.188602 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:38.188688 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:38.190204 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:38.190982 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:38.192244 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:38.192774 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:38.192904 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:38.195565 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:38.195569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:38.195572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:38.195601 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:38.198751 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:38.199581 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:38.199586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:38.199589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:38.199752 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:38.200275 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:38.200281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:38.200284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:38.200669 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:38.202120 (+ 1451us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:38.202280 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 03:07:38.202535 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 03:07:38.213743 4079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:38.222806 4079 catalog_manager.cc:1347] Generated new cluster ID: ea67d86a795a41f6be91f4047a86d29e
I20260807 03:07:38.222934 4079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:38.229398 4079 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:38.231135 4079 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:38.247181 4079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94e5b6d88d3a4a80b7162ce83a599300: Generated new TSK 0
I20260807 03:07:38.248087 4079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:38.254231 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 3961
I20260807 03:07:38.255237 4050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:38.341869 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 3961
2026-08-07T03:07:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2082 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:38 dist-test-slave-fgb3 krb5kdc[4100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:38 dist-test-slave-fgb3 krb5kdc[4100](info): set up 2 sockets
Aug 07 03:07:38 dist-test-slave-fgb3 krb5kdc[4100](info): commencing operation
krb5kdc: starting...
W20260807 03:07:39.527637 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:39 dist-test-slave-fgb3 krb5kdc[4100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:39.815558 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37927
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:40.140570 4122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:40.140858 4122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:40.141000 4122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:40.147977 4122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:40.148087 4122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:40.148164 4122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:40.148247 4122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:40.148304 4122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:40.157519 4122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:40.158711 4122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:40.159937 4122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:40.170696 4128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:40.170738 4129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:40.170925 4131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:40.171567 4122 server_base.cc:1034] running on GCE node
I20260807 03:07:40.172260 4122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:40.173485 4122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:40.174674 4122 hybrid_clock.cc:648] HybridClock initialized: now 1786072060174629 us; error 36 us; skew 500 ppm
I20260807 03:07:40.174989 4122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:40 dist-test-slave-fgb3 krb5kdc[4100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072060, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:40.177831 4122 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:40.179236 4122 webserver.cc:460] Webserver started at http://127.0.0.1:33187/ using document root <none> and password file <none>
I20260807 03:07:40.179858 4122 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:40.179975 4122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:40.180246 4122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:40.183043 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "0c24370bc0604ab79fe42600dc9c223f"
format_stamp: "Formatted at 2026-08-07 03:07:40 on dist-test-slave-fgb3"
server_key: "0cf57a4a8cdccb2bde7721ef83b28d9b"
server_key_iv: "93eb7b46b98436e0c7393503310b9642"
server_key_version: "encryptionkey@0"
I20260807 03:07:40.183729 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "0c24370bc0604ab79fe42600dc9c223f"
format_stamp: "Formatted at 2026-08-07 03:07:40 on dist-test-slave-fgb3"
server_key: "0cf57a4a8cdccb2bde7721ef83b28d9b"
server_key_iv: "93eb7b46b98436e0c7393503310b9642"
server_key_version: "encryptionkey@0"
I20260807 03:07:40.188480 4122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 03:07:40.191701 4143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:40.192876 4122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 03:07:40.193032 4122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "0c24370bc0604ab79fe42600dc9c223f"
format_stamp: "Formatted at 2026-08-07 03:07:40 on dist-test-slave-fgb3"
server_key: "0cf57a4a8cdccb2bde7721ef83b28d9b"
server_key_iv: "93eb7b46b98436e0c7393503310b9642"
server_key_version: "encryptionkey@0"
I20260807 03:07:40.193195 4122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:40.212404 4122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:40.216166 4122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:40.216431 4122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:40.233510 4122 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37927
I20260807 03:07:40.233551 4217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37927 every 8 connection(s)
I20260807 03:07:40.234767 4122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:40.238171 4218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:40.241815 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 4122
I20260807 03:07:40.241971 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:40.242398 5608 external_mini_cluster.cc:1428] Setting key 26df5060a6f6e101f45d0bc5a998a7b1
I20260807 03:07:40.250952 4218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f: Bootstrap starting.
I20260807 03:07:40.254558 4218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:40.255831 4218 log.cc:824] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:40.256227 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:40.244030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47177 (local address 127.0.0.1:37927)
0807 03:07:40.244807 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:40.244822 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:40.244870 (+ 48us) server_negotiation.cc:407] Connection header received
0807 03:07:40.246663 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:40.246690 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:40.247181 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:40.247603 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:40.248085 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:40.250025 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:40.251423 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:40.252374 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:40.252591 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:40.253590 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:40.253601 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:40.253618 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:40.254282 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:40.254389 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:40.254423 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:40.254540 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:40.254703 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 03:07:40.254992 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":321,"threads_started":1}
I20260807 03:07:40.258875 4218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f: No bootstrap required, opened a new log
I20260807 03:07:40.259934 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:40.267740 4218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c24370bc0604ab79fe42600dc9c223f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37927 } }
I20260807 03:07:40.268177 4218 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:40.268285 4218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c24370bc0604ab79fe42600dc9c223f, State: Initialized, Role: FOLLOWER
I20260807 03:07:40.268836 4218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c24370bc0604ab79fe42600dc9c223f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37927 } }
I20260807 03:07:40.269059 4218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:40.269177 4218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:40.269308 4218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:40.272042 4218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c24370bc0604ab79fe42600dc9c223f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37927 } }
I20260807 03:07:40.272255 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:40.264860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49669 (local address 127.0.0.1:37927)
0807 03:07:40.265023 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:40.265029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:40.266809 (+ 1780us) server_negotiation.cc:407] Connection header received
0807 03:07:40.266945 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:40.266950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:40.267017 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:40.267117 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:40.267573 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:40.268572 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:40.269901 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:40.270620 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:40.270755 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:40.271612 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:40.271618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:40.271623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:40.271641 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:40.271681 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:40.271699 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:40.271789 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:40.271858 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 03:07:40.271938 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 03:07:40.272567 4218 leader_election.cc:302] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c24370bc0604ab79fe42600dc9c223f; no voters:
I20260807 03:07:40.273185 4218 leader_election.cc:288] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:40.273263 4230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:40.274307 4230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [term 1 LEADER]: Becoming Leader. State: Replica: 0c24370bc0604ab79fe42600dc9c223f, State: Running, Role: LEADER
I20260807 03:07:40.274803 4230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c24370bc0604ab79fe42600dc9c223f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37927 } }
I20260807 03:07:40.275110 4218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:40.277304 4231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c24370bc0604ab79fe42600dc9c223f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c24370bc0604ab79fe42600dc9c223f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37927 } } }
I20260807 03:07:40.277360 4234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c24370bc0604ab79fe42600dc9c223f. Latest consensus state: current_term: 1 leader_uuid: "0c24370bc0604ab79fe42600dc9c223f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c24370bc0604ab79fe42600dc9c223f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37927 } } }
I20260807 03:07:40.277704 4231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:40.277734 4234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:40.280277 4241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:40.286243 4241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:40.294251 4241 catalog_manager.cc:1347] Generated new cluster ID: b2fd07f5cd54431c8fb01373b1bfbe9e
I20260807 03:07:40.294380 4241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:40.308776 4241 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:40.309921 4241 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:40.317567 4241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c24370bc0604ab79fe42600dc9c223f: Generated new TSK 0
I20260807 03:07:40.318239 4241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:40.330801 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4122
W20260807 03:07:40.331938 4212 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47177
W20260807 03:07:40.332593 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4122: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:40.332724 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 4122
2026-08-07T03:07:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1992 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:40 dist-test-slave-fgb3 krb5kdc[4258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:40 dist-test-slave-fgb3 krb5kdc[4258](info): set up 2 sockets
Aug 07 03:07:40 dist-test-slave-fgb3 krb5kdc[4258](info): commencing operation
krb5kdc: starting...
W20260807 03:07:41.523206 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:41 dist-test-slave-fgb3 krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:41.820969 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:42.143604 4278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:42.143887 4278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:42.144032 4278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:42.151098 4278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:42.151216 4278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:42.151292 4278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:42.151376 4278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:42.151434 4278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:42.161055 4278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:42.162334 4278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:42.163614 4278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:42.174844 4287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:42.174819 4290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:42.175035 4292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:42.175701 4278 server_base.cc:1034] running on GCE node
I20260807 03:07:42.176431 4278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:42.177778 4278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:42.178992 4278 hybrid_clock.cc:648] HybridClock initialized: now 1786072062178943 us; error 43 us; skew 500 ppm
I20260807 03:07:42.179338 4278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:42 dist-test-slave-fgb3 krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:42.185662 4278 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:42.187198 4278 webserver.cc:460] Webserver started at http://127.0.0.1:34479/ using document root <none> and password file <none>
I20260807 03:07:42.187898 4278 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:42.188023 4278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:42.188321 4278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:42.191411 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "42fbff16d38d4c2b9b052077a0f8ca0f"
format_stamp: "Formatted at 2026-08-07 03:07:42 on dist-test-slave-fgb3"
server_key: "ee5772b7aa7feef1f1cc53019d996e54"
server_key_iv: "023d7e362502cb3d781f5f2b852d1adf"
server_key_version: "encryptionkey@0"
I20260807 03:07:42.192162 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "42fbff16d38d4c2b9b052077a0f8ca0f"
format_stamp: "Formatted at 2026-08-07 03:07:42 on dist-test-slave-fgb3"
server_key: "ee5772b7aa7feef1f1cc53019d996e54"
server_key_iv: "023d7e362502cb3d781f5f2b852d1adf"
server_key_version: "encryptionkey@0"
I20260807 03:07:42.197300 4278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:07:42.200709 4299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:42.201967 4278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 03:07:42.202133 4278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "42fbff16d38d4c2b9b052077a0f8ca0f"
format_stamp: "Formatted at 2026-08-07 03:07:42 on dist-test-slave-fgb3"
server_key: "ee5772b7aa7feef1f1cc53019d996e54"
server_key_iv: "023d7e362502cb3d781f5f2b852d1adf"
server_key_version: "encryptionkey@0"
I20260807 03:07:42.202340 4278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:42.208195 4278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:42.212005 4278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:42.212282 4278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:42.229677 4278 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40863
I20260807 03:07:42.229715 4372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40863 every 8 connection(s)
I20260807 03:07:42.231101 4278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:42.234668 4374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:42.238559 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 4278
I20260807 03:07:42.238725 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:42.239216 5608 external_mini_cluster.cc:1428] Setting key c47d589d8055c4dbdbe6792bb7b3447e
I20260807 03:07:42.245088 4374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f: Bootstrap starting.
I20260807 03:07:42.247236 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:42.240955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48673 (local address 127.0.0.1:40863)
0807 03:07:42.241771 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:42.241787 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:42.241830 (+ 43us) server_negotiation.cc:407] Connection header received
0807 03:07:42.243853 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:42.243876 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:42.244194 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:42.244517 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:42.245606 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:42.245613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:42.245625 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:42.245646 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:42.245705 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:42.245726 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:42.245909 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:42.246027 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 03:07:42.246207 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":374,"threads_started":1}
I20260807 03:07:42.248487 4374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:42.249557 4374 log.cc:824] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:42.250998 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:42.252302 4374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f: No bootstrap required, opened a new log
I20260807 03:07:42.260314 4374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260807 03:07:42.260849 4374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:42.260948 4374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fbff16d38d4c2b9b052077a0f8ca0f, State: Initialized, Role: FOLLOWER
I20260807 03:07:42.261767 4374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260807 03:07:42.262117 4374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:42.262291 4374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:42.262507 4374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:42.263428 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:42.260070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47155 (local address 127.0.0.1:40863)
0807 03:07:42.260258 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:42.260265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:42.261079 (+ 814us) server_negotiation.cc:407] Connection header received
0807 03:07:42.261256 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:42.261263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:42.261331 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:42.261431 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:42.262588 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:42.262593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:42.262598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:42.262649 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:42.262715 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:42.262748 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:42.262872 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:42.263021 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 03:07:42.263086 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:07:42.265933 4374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260807 03:07:42.266618 4374 leader_election.cc:302] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42fbff16d38d4c2b9b052077a0f8ca0f; no voters:
I20260807 03:07:42.268090 4374 leader_election.cc:288] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:42.268214 4386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:42.268879 4386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [term 1 LEADER]: Becoming Leader. State: Replica: 42fbff16d38d4c2b9b052077a0f8ca0f, State: Running, Role: LEADER
I20260807 03:07:42.269482 4386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260807 03:07:42.270220 4374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:42.273249 4388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42fbff16d38d4c2b9b052077a0f8ca0f. Latest consensus state: current_term: 1 leader_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } } }
I20260807 03:07:42.273226 4387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fbff16d38d4c2b9b052077a0f8ca0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } } }
I20260807 03:07:42.273715 4387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:42.273715 4388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:42.274636 4392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:42.280095 4392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:42.298250 4392 catalog_manager.cc:1347] Generated new cluster ID: 33db96d4e56444588c5b3a242d5de907
I20260807 03:07:42.298405 4392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:42.311487 4392 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:42.311932 4392 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:42.329734 4392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42fbff16d38d4c2b9b052077a0f8ca0f: Generated new TSK 0
I20260807 03:07:42.330495 4392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:42.337235 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4278
W20260807 03:07:42.338157 4363 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48673
W20260807 03:07:42.338814 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4278: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:42.338927 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 4278
2026-08-07T03:07:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2006 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:42 dist-test-slave-fgb3 krb5kdc[4412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:42 dist-test-slave-fgb3 krb5kdc[4412](info): set up 2 sockets
Aug 07 03:07:42 dist-test-slave-fgb3 krb5kdc[4412](info): commencing operation
krb5kdc: starting...
W20260807 03:07:43.532491 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:43 dist-test-slave-fgb3 krb5kdc[4412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:43.801141 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:44.120405 4435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:44.120705 4435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:44.120862 4435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:44.128026 4435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:44.128149 4435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:44.128228 4435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:44.128321 4435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:44.128378 4435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:44.138108 4435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:44.139382 4435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:44.140645 4435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:44.151718 4445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:44.151793 4446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:44.151759 4449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:44.152642 4435 server_base.cc:1034] running on GCE node
I20260807 03:07:44.153370 4435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:44.154855 4435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:44.156067 4435 hybrid_clock.cc:648] HybridClock initialized: now 1786072064156019 us; error 35 us; skew 500 ppm
I20260807 03:07:44.156427 4435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:44 dist-test-slave-fgb3 krb5kdc[4412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072064, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:44.159409 4435 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:44.160776 4435 webserver.cc:460] Webserver started at http://127.0.0.1:39749/ using document root <none> and password file <none>
I20260807 03:07:44.161453 4435 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:44.161568 4435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:44.161857 4435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:44.164860 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "dbdda2d4859b4624a6d599af77494883"
format_stamp: "Formatted at 2026-08-07 03:07:44 on dist-test-slave-fgb3"
server_key: "79580909c32882e520b32dd854401b09"
server_key_iv: "7e73c087c4832c8df34b2e0b3f3bd6c1"
server_key_version: "encryptionkey@0"
I20260807 03:07:44.165602 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "dbdda2d4859b4624a6d599af77494883"
format_stamp: "Formatted at 2026-08-07 03:07:44 on dist-test-slave-fgb3"
server_key: "79580909c32882e520b32dd854401b09"
server_key_iv: "7e73c087c4832c8df34b2e0b3f3bd6c1"
server_key_version: "encryptionkey@0"
I20260807 03:07:44.170833 4435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 03:07:44.174014 4464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:44.175257 4435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:07:44.175407 4435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "dbdda2d4859b4624a6d599af77494883"
format_stamp: "Formatted at 2026-08-07 03:07:44 on dist-test-slave-fgb3"
server_key: "79580909c32882e520b32dd854401b09"
server_key_iv: "7e73c087c4832c8df34b2e0b3f3bd6c1"
server_key_version: "encryptionkey@0"
I20260807 03:07:44.175561 4435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:44.188835 4435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:44.192437 4435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:44.192665 4435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:44.209650 4435 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38773
I20260807 03:07:44.209668 4534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38773 every 8 connection(s)
I20260807 03:07:44.210987 4435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:44.214637 4535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:44.214962 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 4435
I20260807 03:07:44.215121 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:44.215577 5608 external_mini_cluster.cc:1428] Setting key 53722323e902a8cf0a9907f27e6a3123
I20260807 03:07:44.227618 4535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883: Bootstrap starting.
Aug 07 03:07:44 dist-test-slave-fgb3 krb5kdc[4412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 03:07:44.231256 4535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:44.232312 4535 log.cc:824] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:44.235239 4535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883: No bootstrap required, opened a new log
I20260807 03:07:44.238869 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:44.217254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:48349 (local address 127.0.0.1:38773)
0807 03:07:44.218015 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:44.218027 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:44.218067 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:07:44.220331 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:44.220353 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:44.220728 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:44.221050 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:44.222876 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:44.224168 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:44.225905 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:44.226604 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:44.226723 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:44.231583 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:44.231593 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:44.231610 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:44.231647 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:44.234735 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:44.235596 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:44.235603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:44.235608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:44.235792 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:44.236277 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:44.236280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:44.236283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:44.236767 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:44.237020 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:44.237516 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 03:07:44.237833 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":327,"threads_started":1}
I20260807 03:07:44.242432 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:44.243300 4535 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdda2d4859b4624a6d599af77494883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260807 03:07:44.243629 4535 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:44.243718 4535 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbdda2d4859b4624a6d599af77494883, State: Initialized, Role: FOLLOWER
I20260807 03:07:44.244246 4535 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdda2d4859b4624a6d599af77494883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260807 03:07:44.244467 4535 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:44.244588 4535 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:44.244750 4535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:44.247311 4535 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdda2d4859b4624a6d599af77494883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260807 03:07:44.247756 4535 leader_election.cc:302] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbdda2d4859b4624a6d599af77494883; no voters:
I20260807 03:07:44.248309 4535 leader_election.cc:288] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:44.248421 4550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:44.250526 4535 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:44.250591 4550 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [term 1 LEADER]: Becoming Leader. State: Replica: dbdda2d4859b4624a6d599af77494883, State: Running, Role: LEADER
I20260807 03:07:44.251145 4550 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdda2d4859b4624a6d599af77494883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260807 03:07:44.253839 4553 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbdda2d4859b4624a6d599af77494883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdda2d4859b4624a6d599af77494883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } } }
I20260807 03:07:44.254184 4553 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:44.260689 4554 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbdda2d4859b4624a6d599af77494883. Latest consensus state: current_term: 1 leader_uuid: "dbdda2d4859b4624a6d599af77494883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdda2d4859b4624a6d599af77494883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } } }
I20260807 03:07:44.261015 4554 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:44.261518 4562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:44.262534 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:44.246620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:50165 (local address 127.0.0.1:38773)
0807 03:07:44.246774 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:44.246780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:44.247152 (+ 372us) server_negotiation.cc:407] Connection header received
0807 03:07:44.247302 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:44.247308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:44.247377 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:44.247454 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:44.248910 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:44.249788 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:44.251306 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:44.251872 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:44.252010 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:44.255262 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:44.255269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:44.255275 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:44.255316 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:44.258968 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:44.259915 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:44.259920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:44.259924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:44.260096 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:44.260634 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:44.260639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:44.260642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:44.261097 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:44.261334 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:44.261977 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 03:07:44.262153 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 03:07:44.266666 4562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:44.275872 4562 catalog_manager.cc:1347] Generated new cluster ID: 45bf1d5a6482467c8cdaba803e89a79f
I20260807 03:07:44.275967 4562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:44.286561 4562 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:44.287696 4562 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:44.294596 4562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbdda2d4859b4624a6d599af77494883: Generated new TSK 0
I20260807 03:07:44.295320 4562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:44.300503 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4435
I20260807 03:07:44.302071 4529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:44.399457 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 4435
W20260807 03:07:44.417863 4410 connection.cc:535] client connection to 127.0.0.1:38773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38773): Connection reset by peer (error 104)
2026-08-07T03:07:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:44 dist-test-slave-fgb3 krb5kdc[4581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:44 dist-test-slave-fgb3 krb5kdc[4581](info): set up 2 sockets
Aug 07 03:07:44 dist-test-slave-fgb3 krb5kdc[4581](info): commencing operation
krb5kdc: starting...
W20260807 03:07:45.611351 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:45 dist-test-slave-fgb3 krb5kdc[4581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:45.921420 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:46.333374 4598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:46.333760 4598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:46.333925 4598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:46.343216 4598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:46.343321 4598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:46.343374 4598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:46.343437 4598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:46.343487 4598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:46.357306 4598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:46.359192 4598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:46.360960 4598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:46.374483 4606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:46.374723 4608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:46.376806 4598 server_base.cc:1034] running on GCE node
W20260807 03:07:46.377602 4610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:46.378530 4598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:46.379985 4598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:46.381256 4598 hybrid_clock.cc:648] HybridClock initialized: now 1786072066381190 us; error 47 us; skew 500 ppm
I20260807 03:07:46.381650 4598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:46 dist-test-slave-fgb3 krb5kdc[4581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072066, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:46.385919 4598 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:46.387925 4598 webserver.cc:460] Webserver started at http://127.0.0.1:46719/ using document root <none> and password file <none>
I20260807 03:07:46.388901 4598 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:46.389084 4598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:46.389500 4598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:46.393754 4598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "dd3d9e1820844946b83b7dd4c0b9c309"
format_stamp: "Formatted at 2026-08-07 03:07:46 on dist-test-slave-fgb3"
server_key: "cbfff519c9ee4c18bbab36fc10eb1f0c"
server_key_iv: "8b91c6bb240e31111307018ad2514044"
server_key_version: "encryptionkey@0"
I20260807 03:07:46.394804 4598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "dd3d9e1820844946b83b7dd4c0b9c309"
format_stamp: "Formatted at 2026-08-07 03:07:46 on dist-test-slave-fgb3"
server_key: "cbfff519c9ee4c18bbab36fc10eb1f0c"
server_key_iv: "8b91c6bb240e31111307018ad2514044"
server_key_version: "encryptionkey@0"
I20260807 03:07:46.402032 4598 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.003s
I20260807 03:07:46.406821 4621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:46.408546 4598 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260807 03:07:46.408771 4598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "dd3d9e1820844946b83b7dd4c0b9c309"
format_stamp: "Formatted at 2026-08-07 03:07:46 on dist-test-slave-fgb3"
server_key: "cbfff519c9ee4c18bbab36fc10eb1f0c"
server_key_iv: "8b91c6bb240e31111307018ad2514044"
server_key_version: "encryptionkey@0"
I20260807 03:07:46.408972 4598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:46.434862 4598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:46.439205 4598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:46.439478 4598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:46.459408 4598 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44703
I20260807 03:07:46.459434 4700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44703 every 8 connection(s)
I20260807 03:07:46.461356 4598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:46.466476 4701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:46.470745 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 4598
I20260807 03:07:46.470947 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:46.471517 5608 external_mini_cluster.cc:1428] Setting key e1d5df33e3c4663291811cd63ac13526
I20260807 03:07:46.482048 4701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309: Bootstrap starting.
I20260807 03:07:46.486675 4701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:46.488008 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:46.473383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:42711 (local address 127.0.0.1:44703)
0807 03:07:46.474332 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:46.474350 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:46.474411 (+ 61us) server_negotiation.cc:407] Connection header received
0807 03:07:46.476826 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:46.476857 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:46.477375 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:46.478490 (+ 1115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:46.479022 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:46.480486 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:46.482438 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:46.483186 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:46.483364 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:46.484722 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:46.484732 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:46.484749 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:46.485477 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:46.485568 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:46.485596 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:46.486296 (+ 700us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:46.486492 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 03:07:46.486665 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":388,"threads_started":1}
I20260807 03:07:46.488047 4701 log.cc:824] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:46.492004 4701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309: No bootstrap required, opened a new log
I20260807 03:07:46.492103 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:46.502818 4701 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260807 03:07:46.503432 4701 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:46.503592 4701 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3d9e1820844946b83b7dd4c0b9c309, State: Initialized, Role: FOLLOWER
I20260807 03:07:46.504338 4701 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260807 03:07:46.504716 4701 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:46.504904 4701 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:46.505108 4701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:46.505749 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:46.497493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:45841 (local address 127.0.0.1:44703)
0807 03:07:46.497679 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:46.497684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:46.498015 (+ 331us) server_negotiation.cc:407] Connection header received
0807 03:07:46.498211 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:46.498216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:46.498296 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:46.498486 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:46.499178 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:46.500358 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:46.502476 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:46.503117 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:46.503216 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:46.504774 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:46.504780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:46.504785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:46.504808 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:46.504855 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:46.504874 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:46.504957 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:46.505345 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 03:07:46.505404 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 03:07:46.508638 4701 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260807 03:07:46.509238 4701 leader_election.cc:302] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd3d9e1820844946b83b7dd4c0b9c309; no voters:
I20260807 03:07:46.509958 4701 leader_election.cc:288] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:46.510075 4713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:46.512291 4701 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:46.514400 4713 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [term 1 LEADER]: Becoming Leader. State: Replica: dd3d9e1820844946b83b7dd4c0b9c309, State: Running, Role: LEADER
I20260807 03:07:46.515062 4713 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260807 03:07:46.522456 4714 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } } }
I20260807 03:07:46.522392 4715 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd3d9e1820844946b83b7dd4c0b9c309. Latest consensus state: current_term: 1 leader_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3d9e1820844946b83b7dd4c0b9c309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } } }
I20260807 03:07:46.522837 4714 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:46.522837 4715 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:46.530638 4723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:46.535795 4723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:46.552901 4723 catalog_manager.cc:1347] Generated new cluster ID: 0d0f9bc5efa545cb87ad19c10beb917a
I20260807 03:07:46.553109 4723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:46.567416 4723 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:46.569325 4723 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:46.580103 4723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd3d9e1820844946b83b7dd4c0b9c309: Generated new TSK 0
I20260807 03:07:46.582466 4723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:46.601209 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4598
W20260807 03:07:46.602440 4692 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.138:42711
W20260807 03:07:46.603868 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4598: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:46.604004 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 4598
W20260807 03:07:46.635071 4579 connection.cc:535] client connection to 127.0.0.1:44703 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44703): Connection reset by peer (error 104)
2026-08-07T03:07:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2219 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:46 dist-test-slave-fgb3 krb5kdc[4745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:46 dist-test-slave-fgb3 krb5kdc[4745](info): set up 2 sockets
Aug 07 03:07:46 dist-test-slave-fgb3 krb5kdc[4745](info): commencing operation
krb5kdc: starting...
W20260807 03:07:47.966670 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:48 dist-test-slave-fgb3 krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:48.284722 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:48.606978 4763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:48.607267 4763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:48.607412 4763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:48.615020 4763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:48.615157 4763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:48.615239 4763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:48.615337 4763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:48.615432 4763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:48.629271 4763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:48.630671 4763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:48.631970 4763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:48.643286 4770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:48.643708 4769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:48.643859 4772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:48.644483 4763 server_base.cc:1034] running on GCE node
I20260807 03:07:48.645207 4763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:48.646571 4763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:48.647770 4763 hybrid_clock.cc:648] HybridClock initialized: now 1786072068647714 us; error 50 us; skew 500 ppm
I20260807 03:07:48.648100 4763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:48 dist-test-slave-fgb3 krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072068, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:48.653710 4763 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:48.655254 4763 webserver.cc:460] Webserver started at http://127.0.0.1:40993/ using document root <none> and password file <none>
I20260807 03:07:48.655913 4763 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:48.656020 4763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:48.656313 4763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:48.659317 4763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "979dbfbf33ff43f094a44d5463c25dca"
format_stamp: "Formatted at 2026-08-07 03:07:48 on dist-test-slave-fgb3"
server_key: "f6b1bd712e86d317fe3cd3c92aac0b52"
server_key_iv: "18aba906e8e529a29e92cd5fc4e737f3"
server_key_version: "encryptionkey@0"
I20260807 03:07:48.660044 4763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "979dbfbf33ff43f094a44d5463c25dca"
format_stamp: "Formatted at 2026-08-07 03:07:48 on dist-test-slave-fgb3"
server_key: "f6b1bd712e86d317fe3cd3c92aac0b52"
server_key_iv: "18aba906e8e529a29e92cd5fc4e737f3"
server_key_version: "encryptionkey@0"
I20260807 03:07:48.665287 4763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:07:48.668744 4780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:48.670120 4763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 03:07:48.670280 4763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "979dbfbf33ff43f094a44d5463c25dca"
format_stamp: "Formatted at 2026-08-07 03:07:48 on dist-test-slave-fgb3"
server_key: "f6b1bd712e86d317fe3cd3c92aac0b52"
server_key_iv: "18aba906e8e529a29e92cd5fc4e737f3"
server_key_version: "encryptionkey@0"
I20260807 03:07:48.670496 4763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:48.676059 4763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:48.680136 4763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:48.680404 4763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:48.698848 4763 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41161
I20260807 03:07:48.698890 4852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41161 every 8 connection(s)
I20260807 03:07:48.700244 4763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:48.704067 4854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:48.704927 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 4763
I20260807 03:07:48.705111 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:48.705703 5608 external_mini_cluster.cc:1428] Setting key dc9b975b04acf93dd416f9e300862178
I20260807 03:07:48.714489 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:48.707650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:53403 (local address 127.0.0.1:41161)
0807 03:07:48.708509 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:48.708524 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:48.708580 (+ 56us) server_negotiation.cc:407] Connection header received
0807 03:07:48.710505 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:48.710536 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:48.710985 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:48.711444 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:48.712428 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:48.712437 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:48.712454 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:48.712470 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:48.712542 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:48.712569 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:48.712774 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:48.712895 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 03:07:48.713271 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":320,"threads_started":1}
I20260807 03:07:48.718394 4854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca: Bootstrap starting.
I20260807 03:07:48.718538 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:48.721207 4854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:48.722277 4854 log.cc:824] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:48.725456 4854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca: No bootstrap required, opened a new log
I20260807 03:07:48.727849 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:48.723856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:37287 (local address 127.0.0.1:41161)
0807 03:07:48.724022 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:48.724028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:48.726071 (+ 2043us) server_negotiation.cc:407] Connection header received
0807 03:07:48.726224 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:48.726228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:48.726270 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:48.726378 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:48.727214 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:48.727220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:48.727225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:48.727242 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:48.727284 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:48.727299 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:48.727405 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:48.727509 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 03:07:48.727569 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:07:48.735785 4854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979dbfbf33ff43f094a44d5463c25dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 03:07:48.736191 4854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:48.736290 4854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979dbfbf33ff43f094a44d5463c25dca, State: Initialized, Role: FOLLOWER
I20260807 03:07:48.736904 4854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979dbfbf33ff43f094a44d5463c25dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 03:07:48.737167 4854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:48.737296 4854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:48.737445 4854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:48.740480 4854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979dbfbf33ff43f094a44d5463c25dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 03:07:48.741026 4854 leader_election.cc:302] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 979dbfbf33ff43f094a44d5463c25dca; no voters:
I20260807 03:07:48.741629 4854 leader_election.cc:288] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:48.741719 4872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:48.742973 4872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [term 1 LEADER]: Becoming Leader. State: Replica: 979dbfbf33ff43f094a44d5463c25dca, State: Running, Role: LEADER
I20260807 03:07:48.743569 4872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979dbfbf33ff43f094a44d5463c25dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 03:07:48.743795 4854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:48.746047 4873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "979dbfbf33ff43f094a44d5463c25dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979dbfbf33ff43f094a44d5463c25dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } } }
I20260807 03:07:48.746440 4873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:48.749383 4874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 979dbfbf33ff43f094a44d5463c25dca. Latest consensus state: current_term: 1 leader_uuid: "979dbfbf33ff43f094a44d5463c25dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979dbfbf33ff43f094a44d5463c25dca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } } }
I20260807 03:07:48.750180 4874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:48.759836 4882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:48.764560 4882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:48.772920 4882 catalog_manager.cc:1347] Generated new cluster ID: 15cd953ab74641d3b11532106ebb3e57
I20260807 03:07:48.773015 4882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:48.800052 4882 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:48.800447 4882 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:48.806203 4882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 979dbfbf33ff43f094a44d5463c25dca: Generated new TSK 0
I20260807 03:07:48.806874 4882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:48.817137 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4763
W20260807 03:07:48.818532 4846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.138:53403
W20260807 03:07:48.819290 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4763: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:48.819424 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 4763
2026-08-07T03:07:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2208 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:48 dist-test-slave-fgb3 krb5kdc[4908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:48 dist-test-slave-fgb3 krb5kdc[4908](info): set up 2 sockets
Aug 07 03:07:48 dist-test-slave-fgb3 krb5kdc[4908](info): commencing operation
krb5kdc: starting...
W20260807 03:07:50.028403 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:50 dist-test-slave-fgb3 krb5kdc[4908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
W20260807 03:07:50.566370 5608 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.0.1: real 0.267s user 0.000s sys 0.026s
I20260807 03:07:50.567144 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:50.902817 4924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:50.903126 4924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:50.903268 4924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:50.910696 4924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:50.910820 4924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:50.910907 4924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:50.910997 4924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:50.911060 4924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:50.921087 4924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:50.922372 4924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:50.923893 4924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:50.935928 4930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:50.936046 4932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:50.935945 4929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:50.937783 4924 server_base.cc:1034] running on GCE node
I20260807 03:07:50.938752 4924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:50.940486 4924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:50.941730 4924 hybrid_clock.cc:648] HybridClock initialized: now 1786072070941652 us; error 62 us; skew 500 ppm
I20260807 03:07:50.942137 4924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:50 dist-test-slave-fgb3 krb5kdc[4908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:50.945923 4924 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:50.947636 4924 webserver.cc:460] Webserver started at http://127.0.0.1:39955/ using document root <none> and password file <none>
I20260807 03:07:50.948509 4924 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:50.948640 4924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:50.948948 4924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:50.953197 4924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "8f7dfc56468749a3a293c4e72d8fcc6a"
format_stamp: "Formatted at 2026-08-07 03:07:50 on dist-test-slave-fgb3"
server_key: "b307b66eea863aa73fc6bf79b5038e76"
server_key_iv: "4e19e5340381fa8484feb53ad95edd7f"
server_key_version: "encryptionkey@0"
I20260807 03:07:50.954212 4924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "8f7dfc56468749a3a293c4e72d8fcc6a"
format_stamp: "Formatted at 2026-08-07 03:07:50 on dist-test-slave-fgb3"
server_key: "b307b66eea863aa73fc6bf79b5038e76"
server_key_iv: "4e19e5340381fa8484feb53ad95edd7f"
server_key_version: "encryptionkey@0"
I20260807 03:07:50.960947 4924 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 03:07:50.965268 4944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:50.966782 4924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 03:07:50.966961 4924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "8f7dfc56468749a3a293c4e72d8fcc6a"
format_stamp: "Formatted at 2026-08-07 03:07:50 on dist-test-slave-fgb3"
server_key: "b307b66eea863aa73fc6bf79b5038e76"
server_key_iv: "4e19e5340381fa8484feb53ad95edd7f"
server_key_version: "encryptionkey@0"
I20260807 03:07:50.967145 4924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:50.984601 4924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:50.989506 4924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:50.989843 4924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:51.009323 4924 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42695
I20260807 03:07:51.009325 5011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42695 every 8 connection(s)
I20260807 03:07:51.011063 4924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:51.014863 5013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:51.018491 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 4924
I20260807 03:07:51.018679 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:51.019235 5608 external_mini_cluster.cc:1428] Setting key 992d9c44c0ac108d15ec95539f29a45c
I20260807 03:07:51.028359 5013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a: Bootstrap starting.
I20260807 03:07:51.032078 5013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:51.033160 5013 log.cc:824] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:51.036592 5013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a: No bootstrap required, opened a new log
Aug 07 03:07:51 dist-test-slave-fgb3 krb5kdc[4908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 03:07:51.046283 5013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260807 03:07:51.046758 5013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:51.046873 5013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7dfc56468749a3a293c4e72d8fcc6a, State: Initialized, Role: FOLLOWER
I20260807 03:07:51.047576 5013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260807 03:07:51.047863 5013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:51.048040 5013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:51.048240 5013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:51.050567 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:51.021052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:33345 (local address 127.0.0.1:42695)
0807 03:07:51.030066 (+ 9014us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:51.030086 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:51.030501 (+ 415us) server_negotiation.cc:407] Connection header received
0807 03:07:51.032935 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:51.032964 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:51.033492 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:51.033946 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:51.035465 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:51.036782 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:51.038046 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:51.038784 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:51.038958 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:51.043017 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:51.043025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:51.043040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:51.043083 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:51.046082 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:51.046941 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:51.046949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:51.046958 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:51.047138 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:51.047640 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:51.047643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:51.047646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:51.048272 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:51.048590 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:51.049142 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 03:07:51.049515 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8705,"thread_start_us":8444,"threads_started":1}
I20260807 03:07:51.051455 5013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260807 03:07:51.052038 5013 leader_election.cc:302] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f7dfc56468749a3a293c4e72d8fcc6a; no voters:
I20260807 03:07:51.052721 5013 leader_election.cc:288] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:51.052860 5020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:51.054675 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:51.059911 5020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [term 1 LEADER]: Becoming Leader. State: Replica: 8f7dfc56468749a3a293c4e72d8fcc6a, State: Running, Role: LEADER
I20260807 03:07:51.060570 5020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260807 03:07:51.067305 5022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } } }
I20260807 03:07:51.067787 5022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:51.068249 5020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f7dfc56468749a3a293c4e72d8fcc6a. Latest consensus state: current_term: 1 leader_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7dfc56468749a3a293c4e72d8fcc6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } } }
I20260807 03:07:51.068498 5020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:51.071985 5013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:51.073199 5028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:51.081511 5028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:51.109515 5028 catalog_manager.cc:1347] Generated new cluster ID: 1f3fa12fe0af4257861b2287da1931f6
I20260807 03:07:51.109726 5028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:51.112953 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:51.080213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:38879 (local address 127.0.0.1:42695)
0807 03:07:51.080405 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:51.080411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:51.095608 (+ 15197us) server_negotiation.cc:407] Connection header received
0807 03:07:51.095795 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:51.095801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:51.095896 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:51.095997 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:51.098174 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:51.099409 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:51.101269 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:51.101941 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:51.102066 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:51.105696 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:51.105704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:51.105710 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:51.105755 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:51.109434 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:51.110430 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:51.110436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:51.110439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:51.110589 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:51.111174 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:51.111179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:51.111183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:51.111596 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:51.111822 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:51.112439 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 03:07:51.112614 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 03:07:51.132531 5028 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:51.134389 5028 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:51.148645 5028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f7dfc56468749a3a293c4e72d8fcc6a: Generated new TSK 0
I20260807 03:07:51.149645 5028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:51.156287 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 4924
I20260807 03:07:51.157629 5005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:51.245963 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 4924
W20260807 03:07:51.267222 4904 connection.cc:535] client connection to 127.0.0.1:42695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42695): Connection reset by peer (error 104)
2026-08-07T03:07:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:51 dist-test-slave-fgb3 krb5kdc[5062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:51 dist-test-slave-fgb3 krb5kdc[5062](info): set up 2 sockets
Aug 07 03:07:51 dist-test-slave-fgb3 krb5kdc[5062](info): commencing operation
krb5kdc: starting...
W20260807 03:07:52.467217 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.009s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:52 dist-test-slave-fgb3 krb5kdc[5062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:52.810861 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34327
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:53.137578 5086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:53.137835 5086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:53.137952 5086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:53.144981 5086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:53.145078 5086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:53.145150 5086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:53.145222 5086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:53.145277 5086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:53.154847 5086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34327
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:53.156051 5086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:53.157297 5086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:53.168175 5092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:53.168283 5095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:53.168700 5091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:53.169661 5086 server_base.cc:1034] running on GCE node
I20260807 03:07:53.170445 5086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:53.171788 5086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:53.172988 5086 hybrid_clock.cc:648] HybridClock initialized: now 1786072073172937 us; error 44 us; skew 500 ppm
I20260807 03:07:53.173303 5086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:53 dist-test-slave-fgb3 krb5kdc[5062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072073, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:53.176297 5086 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:53.177682 5086 webserver.cc:460] Webserver started at http://127.0.0.1:44533/ using document root <none> and password file <none>
I20260807 03:07:53.178356 5086 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:53.178484 5086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:53.178764 5086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:53.181644 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "223d7d7a17a24330980eaf24711388e3"
format_stamp: "Formatted at 2026-08-07 03:07:53 on dist-test-slave-fgb3"
server_key: "fe4bc20dc51e574ab7870a6ff139cfb7"
server_key_iv: "74b0fab78430a7764548b84b5ff1dd71"
server_key_version: "encryptionkey@0"
I20260807 03:07:53.182387 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "223d7d7a17a24330980eaf24711388e3"
format_stamp: "Formatted at 2026-08-07 03:07:53 on dist-test-slave-fgb3"
server_key: "fe4bc20dc51e574ab7870a6ff139cfb7"
server_key_iv: "74b0fab78430a7764548b84b5ff1dd71"
server_key_version: "encryptionkey@0"
I20260807 03:07:53.187363 5086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 03:07:53.190747 5105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:53.192039 5086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 03:07:53.192195 5086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "223d7d7a17a24330980eaf24711388e3"
format_stamp: "Formatted at 2026-08-07 03:07:53 on dist-test-slave-fgb3"
server_key: "fe4bc20dc51e574ab7870a6ff139cfb7"
server_key_iv: "74b0fab78430a7764548b84b5ff1dd71"
server_key_version: "encryptionkey@0"
I20260807 03:07:53.192364 5086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:53.214288 5086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:53.218266 5086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:53.218536 5086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:53.236195 5086 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34327
I20260807 03:07:53.236219 5174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34327 every 8 connection(s)
I20260807 03:07:53.237465 5086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:53.240149 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5086
I20260807 03:07:53.240305 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:53.240747 5608 external_mini_cluster.cc:1428] Setting key d461e827ef347d609dad2045db13e59d
I20260807 03:07:53.241755 5176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:53.252832 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:53.242557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:57877 (local address 127.0.0.1:34327)
0807 03:07:53.243244 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:53.243254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:53.243293 (+ 39us) server_negotiation.cc:407] Connection header received
0807 03:07:53.244840 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:53.244861 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:53.245209 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:53.245568 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:53.246095 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:53.247397 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:53.248706 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:53.249346 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:53.249528 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:53.250689 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:53.250696 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:53.250708 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:53.251188 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:53.251270 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:53.251293 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:53.251438 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:53.251703 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 03:07:53.251880 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":395,"threads_started":1}
I20260807 03:07:53.256282 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:53.256582 5176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3: Bootstrap starting.
I20260807 03:07:53.260305 5176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:53.261699 5176 log.cc:824] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:53.264861 5176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3: No bootstrap required, opened a new log
I20260807 03:07:53.266682 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:53.260628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:46369 (local address 127.0.0.1:34327)
0807 03:07:53.260813 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:53.260820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:53.261304 (+ 484us) server_negotiation.cc:407] Connection header received
0807 03:07:53.261508 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:53.261512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:53.261572 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:53.261659 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:53.262175 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:53.263013 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:53.264337 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:53.264762 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:53.264878 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:53.265930 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:53.265934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:53.265938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:53.265954 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:53.265990 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:53.266005 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:53.266106 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:53.266279 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 03:07:53.266352 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:07:53.274926 5176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "223d7d7a17a24330980eaf24711388e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34327 } }
I20260807 03:07:53.275352 5176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:53.275455 5176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 223d7d7a17a24330980eaf24711388e3, State: Initialized, Role: FOLLOWER
I20260807 03:07:53.276062 5176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "223d7d7a17a24330980eaf24711388e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34327 } }
I20260807 03:07:53.276333 5176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:53.276463 5176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:53.276629 5176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:53.279738 5176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "223d7d7a17a24330980eaf24711388e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34327 } }
I20260807 03:07:53.280361 5176 leader_election.cc:302] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 223d7d7a17a24330980eaf24711388e3; no voters:
I20260807 03:07:53.280956 5176 leader_election.cc:288] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:53.281071 5189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:53.282243 5189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [term 1 LEADER]: Becoming Leader. State: Replica: 223d7d7a17a24330980eaf24711388e3, State: Running, Role: LEADER
I20260807 03:07:53.282899 5189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "223d7d7a17a24330980eaf24711388e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34327 } }
I20260807 03:07:53.283236 5176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:53.285708 5191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 223d7d7a17a24330980eaf24711388e3. Latest consensus state: current_term: 1 leader_uuid: "223d7d7a17a24330980eaf24711388e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "223d7d7a17a24330980eaf24711388e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34327 } } }
I20260807 03:07:53.285864 5190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "223d7d7a17a24330980eaf24711388e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "223d7d7a17a24330980eaf24711388e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34327 } } }
I20260807 03:07:53.286203 5190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:53.286203 5191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:53.301505 5196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:53.306473 5196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:53.317538 5196 catalog_manager.cc:1347] Generated new cluster ID: 4ff2ce04a6b64b538e4529abffd33495
I20260807 03:07:53.317706 5196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:53.339771 5196 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:53.341588 5196 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:53.346285 5212 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 03:07:53.349804 5196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 223d7d7a17a24330980eaf24711388e3: Generated new TSK 0
I20260807 03:07:53.350515 5196 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:53.360066 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5086
W20260807 03:07:53.361162 5169 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.138:57877
W20260807 03:07:53.361949 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5086: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:53.362077 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5086
W20260807 03:07:53.382530 5056 connection.cc:535] client connection to 127.0.0.1:34327 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34327): Connection reset by peer (error 104)
2026-08-07T03:07:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2114 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:53 dist-test-slave-fgb3 krb5kdc[5216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:53 dist-test-slave-fgb3 krb5kdc[5216](info): set up 2 sockets
Aug 07 03:07:53 dist-test-slave-fgb3 krb5kdc[5216](info): commencing operation
krb5kdc: starting...
W20260807 03:07:54.600028 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:54 dist-test-slave-fgb3 krb5kdc[5216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 03:07:54.922539 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38893
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:55.258266 5234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:55.258592 5234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:55.258736 5234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:55.265632 5234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:55.265748 5234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:55.265830 5234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:55.265911 5234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:55.265972 5234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:55.275548 5234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:55.276839 5234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:55.278119 5234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:55.289465 5253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:55.289412 5250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:55.289767 5249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:55.291628 5234 server_base.cc:1034] running on GCE node
I20260807 03:07:55.292333 5234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:55.293656 5234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:55.294864 5234 hybrid_clock.cc:648] HybridClock initialized: now 1786072075294820 us; error 38 us; skew 500 ppm
I20260807 03:07:55.295188 5234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:55 dist-test-slave-fgb3 krb5kdc[5216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072075, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:55.301703 5234 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 03:07:55.303241 5234 webserver.cc:460] Webserver started at http://127.0.0.1:41861/ using document root <none> and password file <none>
I20260807 03:07:55.303884 5234 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:55.303993 5234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:55.304275 5234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:55.307139 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "a177dcdf6e4841aa89035ef5caf8ad11"
format_stamp: "Formatted at 2026-08-07 03:07:55 on dist-test-slave-fgb3"
server_key: "42f3b8800aa950e0274f2f6b700485fd"
server_key_iv: "3e25cad3fc4f87967e7a040e608df6be"
server_key_version: "encryptionkey@0"
I20260807 03:07:55.307844 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "a177dcdf6e4841aa89035ef5caf8ad11"
format_stamp: "Formatted at 2026-08-07 03:07:55 on dist-test-slave-fgb3"
server_key: "42f3b8800aa950e0274f2f6b700485fd"
server_key_iv: "3e25cad3fc4f87967e7a040e608df6be"
server_key_version: "encryptionkey@0"
I20260807 03:07:55.313004 5234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:07:55.317191 5263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:55.318610 5234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:07:55.318825 5234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "a177dcdf6e4841aa89035ef5caf8ad11"
format_stamp: "Formatted at 2026-08-07 03:07:55 on dist-test-slave-fgb3"
server_key: "42f3b8800aa950e0274f2f6b700485fd"
server_key_iv: "3e25cad3fc4f87967e7a040e608df6be"
server_key_version: "encryptionkey@0"
I20260807 03:07:55.319012 5234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:55.324750 5234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:55.328770 5234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:55.329036 5234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:55.348412 5234 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38893
I20260807 03:07:55.348430 5343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38893 every 8 connection(s)
I20260807 03:07:55.349822 5234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:55.353402 5344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:55.357863 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5234
I20260807 03:07:55.358069 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:55.358660 5608 external_mini_cluster.cc:1428] Setting key 68d992aa20837aca0d6505415a2eafd7
I20260807 03:07:55.365350 5344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11: Bootstrap starting.
I20260807 03:07:55.367931 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:55.360508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:46293 (local address 127.0.0.1:38893)
0807 03:07:55.361407 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:55.361424 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:55.361776 (+ 352us) server_negotiation.cc:407] Connection header received
0807 03:07:55.364253 (+ 2477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:55.364280 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:55.364682 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:55.365078 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:55.366053 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:55.366061 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:55.366077 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:55.366099 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:55.366178 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:55.366209 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:55.366437 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:55.366574 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 03:07:55.366746 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":415,"threads_started":1}
I20260807 03:07:55.369163 5344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:55.370179 5344 log.cc:824] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:55.371971 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:55.373149 5344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11: No bootstrap required, opened a new log
I20260807 03:07:55.380507 5344 raft_consensus.cc:348] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38893 } }
I20260807 03:07:55.380980 5344 raft_consensus.cc:374] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:55.381091 5344 raft_consensus.cc:729] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a177dcdf6e4841aa89035ef5caf8ad11, State: Initialized, Role: FOLLOWER
I20260807 03:07:55.381712 5344 consensus_queue.cc:260] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38893 } }
I20260807 03:07:55.381986 5344 raft_consensus.cc:388] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:55.382130 5344 raft_consensus.cc:482] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:55.382362 5344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:55.382936 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:55.380141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:35699 (local address 127.0.0.1:38893)
0807 03:07:55.380310 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:55.380316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:55.380850 (+ 534us) server_negotiation.cc:407] Connection header received
0807 03:07:55.381029 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:55.381034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:55.381096 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:55.381192 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:55.382237 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:55.382242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:55.382246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:55.382265 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:55.382336 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:55.382363 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:55.382487 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:55.382608 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 03:07:55.382668 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:07:55.385149 5344 raft_consensus.cc:504] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38893 } }
I20260807 03:07:55.385584 5344 leader_election.cc:302] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a177dcdf6e4841aa89035ef5caf8ad11; no voters:
I20260807 03:07:55.386111 5344 leader_election.cc:288] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:55.386247 5356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:55.386902 5356 raft_consensus.cc:686] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [term 1 LEADER]: Becoming Leader. State: Replica: a177dcdf6e4841aa89035ef5caf8ad11, State: Running, Role: LEADER
I20260807 03:07:55.387517 5356 consensus_queue.cc:237] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38893 } }
I20260807 03:07:55.387969 5344 sys_catalog.cc:564] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:55.390419 5357 sys_catalog.cc:455] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38893 } } }
I20260807 03:07:55.390831 5360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a177dcdf6e4841aa89035ef5caf8ad11. Latest consensus state: current_term: 1 leader_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a177dcdf6e4841aa89035ef5caf8ad11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38893 } } }
I20260807 03:07:55.391047 5357 sys_catalog.cc:458] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:55.391108 5360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11 [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:55.396262 5364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:55.401684 5364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:55.421762 5364 catalog_manager.cc:1347] Generated new cluster ID: b40916f0aa88477096a954227e1a6082
I20260807 03:07:55.421886 5364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:55.445974 5364 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:55.446698 5364 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:55.459918 5364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a177dcdf6e4841aa89035ef5caf8ad11: Generated new TSK 0
I20260807 03:07:55.460841 5364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:55.470670 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5234
W20260807 03:07:55.472045 5339 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.138:46293
W20260807 03:07:55.472721 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5234: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:55.472868 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5234
2026-08-07T03:07:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:55 dist-test-slave-fgb3 krb5kdc[5387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:55 dist-test-slave-fgb3 krb5kdc[5387](info): set up 2 sockets
Aug 07 03:07:55 dist-test-slave-fgb3 krb5kdc[5387](info): commencing operation
krb5kdc: starting...
W20260807 03:07:56.683530 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:56 dist-test-slave-fgb3 krb5kdc[5387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:07:57.020725 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33635
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:57.373553 5403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:57.373833 5403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:57.373963 5403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:57.381560 5403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:57.381739 5403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:57.381825 5403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:57.381913 5403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:57.381973 5403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:57.392592 5403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:57.393833 5403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:57.395103 5403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:57.406265 5415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:57.406306 5411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:57.406443 5418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:57.407665 5403 server_base.cc:1034] running on GCE node
I20260807 03:07:57.408416 5403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:57.409759 5403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:57.410959 5403 hybrid_clock.cc:648] HybridClock initialized: now 1786072077410904 us; error 44 us; skew 500 ppm
I20260807 03:07:57.411276 5403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:57 dist-test-slave-fgb3 krb5kdc[5387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072077, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:57.414130 5403 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:07:57.415511 5403 webserver.cc:460] Webserver started at http://127.5.122.62:35699/ using document root <none> and password file <none>
I20260807 03:07:57.416152 5403 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:57.416268 5403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:57.416545 5403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:57.419457 5403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "990d1f9984d848b591a94cba3283da0e"
format_stamp: "Formatted at 2026-08-07 03:07:57 on dist-test-slave-fgb3"
server_key: "865a66068a40a392d11482d9d5784330"
server_key_iv: "b008cc145edf94651aab16e1165b70dc"
server_key_version: "encryptionkey@0"
I20260807 03:07:57.420177 5403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "990d1f9984d848b591a94cba3283da0e"
format_stamp: "Formatted at 2026-08-07 03:07:57 on dist-test-slave-fgb3"
server_key: "865a66068a40a392d11482d9d5784330"
server_key_iv: "b008cc145edf94651aab16e1165b70dc"
server_key_version: "encryptionkey@0"
I20260807 03:07:57.425205 5403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:07:57.434958 5426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:57.436889 5403 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.004s
I20260807 03:07:57.437098 5403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "990d1f9984d848b591a94cba3283da0e"
format_stamp: "Formatted at 2026-08-07 03:07:57 on dist-test-slave-fgb3"
server_key: "865a66068a40a392d11482d9d5784330"
server_key_iv: "b008cc145edf94651aab16e1165b70dc"
server_key_version: "encryptionkey@0"
I20260807 03:07:57.437353 5403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:57.462575 5403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:57.466732 5403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:57.466988 5403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:57.484012 5403 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:33635
I20260807 03:07:57.484052 5507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:33635 every 8 connection(s)
I20260807 03:07:57.485253 5403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:57.488780 5508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:57.495416 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5403
I20260807 03:07:57.495580 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:57.495987 5608 external_mini_cluster.cc:1428] Setting key ac704c2ca06a89b8fb3ea8f3ff52691a
I20260807 03:07:57.498873 5508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e: Bootstrap starting.
I20260807 03:07:57.502158 5508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:57.503309 5508 log.cc:824] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:57.506215 5508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e: No bootstrap required, opened a new log
Aug 07 03:07:57 dist-test-slave-fgb3 krb5kdc[5387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:07:57.515964 5508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990d1f9984d848b591a94cba3283da0e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33635 } }
I20260807 03:07:57.516351 5508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:57.516462 5508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990d1f9984d848b591a94cba3283da0e, State: Initialized, Role: FOLLOWER
I20260807 03:07:57.517109 5508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990d1f9984d848b591a94cba3283da0e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33635 } }
I20260807 03:07:57.517408 5508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:57.517565 5508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:57.517748 5508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:57.519436 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:57.497667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38683 (local address 127.5.122.62:33635)
0807 03:07:57.498434 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:57.498452 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:57.498492 (+ 40us) server_negotiation.cc:407] Connection header received
0807 03:07:57.500172 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:57.500195 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:57.500545 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:57.500830 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:57.502615 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:57.504012 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:57.505609 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:57.506229 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:57.506419 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:57.511114 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:57.511125 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:57.511146 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:57.511186 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:57.515020 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:57.515718 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:57.515725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:57.515733 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:57.515936 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:57.516396 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:57.516401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:57.516404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:57.517044 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:57.517399 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:57.517728 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 03:07:57.518073 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":350,"threads_started":1}
I20260807 03:07:57.520602 5508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990d1f9984d848b591a94cba3283da0e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33635 } }
I20260807 03:07:57.521150 5508 leader_election.cc:302] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 990d1f9984d848b591a94cba3283da0e; no voters:
I20260807 03:07:57.521798 5508 leader_election.cc:288] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:57.521890 5514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:57.522516 5514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [term 1 LEADER]: Becoming Leader. State: Replica: 990d1f9984d848b591a94cba3283da0e, State: Running, Role: LEADER
I20260807 03:07:57.523053 5514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990d1f9984d848b591a94cba3283da0e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33635 } }
I20260807 03:07:57.523375 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:57.524951 5508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:57.525776 5517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "990d1f9984d848b591a94cba3283da0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990d1f9984d848b591a94cba3283da0e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33635 } } }
I20260807 03:07:57.526134 5517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:57.528563 5516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 990d1f9984d848b591a94cba3283da0e. Latest consensus state: current_term: 1 leader_uuid: "990d1f9984d848b591a94cba3283da0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990d1f9984d848b591a94cba3283da0e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33635 } } }
I20260807 03:07:57.528863 5516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:57.529572 5521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:57.534061 5521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:57.552670 5521 catalog_manager.cc:1347] Generated new cluster ID: def90caba8d6454cb6d7eb29b6ebff71
I20260807 03:07:57.552793 5521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:57.569751 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:57.538350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41739 (local address 127.5.122.62:33635)
0807 03:07:57.538550 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:57.538556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:57.549856 (+ 11300us) server_negotiation.cc:407] Connection header received
0807 03:07:57.550641 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:57.550647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:57.550724 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:57.550837 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:57.554731 (+ 3894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:57.555897 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:57.557767 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:57.558467 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:57.558563 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:57.561854 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:57.561861 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:57.561866 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:07:57.561902 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:57.565537 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:57.566547 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:57.566553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:57.566556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:57.566714 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:07:57.567274 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:07:57.567279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:07:57.567282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:07:57.567674 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:57.567936 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:57.569192 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0807 03:07:57.569372 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 03:07:57.585638 5521 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:57.586987 5521 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:57.593889 5521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 990d1f9984d848b591a94cba3283da0e: Generated new TSK 0
I20260807 03:07:57.594718 5521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:57.605587 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5403
I20260807 03:07:57.607250 5501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:07:57.699744 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5403
W20260807 03:07:57.721581 5385 connection.cc:535] client connection to 127.5.122.62:33635 recv error: Network error: failed to read from TLS socket (remote: 127.5.122.62:33635): Connection reset by peer (error 104)
2026-08-07T03:07:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:57 dist-test-slave-fgb3 krb5kdc[5558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:57 dist-test-slave-fgb3 krb5kdc[5558](info): set up 2 sockets
Aug 07 03:07:57 dist-test-slave-fgb3 krb5kdc[5558](info): commencing operation
krb5kdc: starting...
W20260807 03:07:58.912160 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:07:59 dist-test-slave-fgb3 krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:07:59.209161 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42321
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:42367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:42321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:07:59.535173 5582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:07:59.535449 5582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:07:59.535578 5582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:07:59.542589 5582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:07:59.542690 5582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:07:59.542755 5582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:07:59.542834 5582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:07:59.542889 5582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:07:59.552469 5582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:42367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:42321
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:42321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:07:59.553687 5582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:07:59.554988 5582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:07:59.566095 5601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:59.566263 5599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:07:59.566226 5596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:07:59.567528 5582 server_base.cc:1034] running on GCE node
I20260807 03:07:59.568292 5582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:07:59.569667 5582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:07:59.570854 5582 hybrid_clock.cc:648] HybridClock initialized: now 1786072079570805 us; error 42 us; skew 500 ppm
I20260807 03:07:59.571179 5582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:07:59 dist-test-slave-fgb3 krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072079, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:07:59.574136 5582 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:07:59.575573 5582 webserver.cc:460] Webserver started at http://127.5.122.62:45437/ using document root <none> and password file <none>
I20260807 03:07:59.576231 5582 fs_manager.cc:359] Metadata directory not provided
I20260807 03:07:59.576346 5582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:07:59.576629 5582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:07:59.579547 5582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "0a9a09ce381343a2a3c53f2dcdde169e"
format_stamp: "Formatted at 2026-08-07 03:07:59 on dist-test-slave-fgb3"
server_key: "a4d67715e282b26c695b75391394fe69"
server_key_iv: "04a385c8647e258b09afa9e861f10533"
server_key_version: "encryptionkey@0"
I20260807 03:07:59.580258 5582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "0a9a09ce381343a2a3c53f2dcdde169e"
format_stamp: "Formatted at 2026-08-07 03:07:59 on dist-test-slave-fgb3"
server_key: "a4d67715e282b26c695b75391394fe69"
server_key_iv: "04a385c8647e258b09afa9e861f10533"
server_key_version: "encryptionkey@0"
I20260807 03:07:59.585178 5582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 03:07:59.588477 5614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:07:59.589695 5582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 03:07:59.589849 5582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "0a9a09ce381343a2a3c53f2dcdde169e"
format_stamp: "Formatted at 2026-08-07 03:07:59 on dist-test-slave-fgb3"
server_key: "a4d67715e282b26c695b75391394fe69"
server_key_iv: "04a385c8647e258b09afa9e861f10533"
server_key_version: "encryptionkey@0"
I20260807 03:07:59.590015 5582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:07:59.616719 5582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:07:59.620838 5582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:07:59.621096 5582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:07:59.638695 5582 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:42321
I20260807 03:07:59.638715 5695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:42321 every 8 connection(s)
I20260807 03:07:59.640148 5582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:07:59.643801 5697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:07:59.648202 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5582
I20260807 03:07:59.648355 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:07:59.648765 5608 external_mini_cluster.cc:1428] Setting key 8efc5d3fc8a8984643715f1339bed443
I20260807 03:07:59.655915 5697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e: Bootstrap starting.
I20260807 03:07:59.659441 5697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e: Neither blocks nor log segments found. Creating new log.
I20260807 03:07:59.660709 5697 log.cc:824] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e: Log is configured to *not* fsync() on all Append() calls
I20260807 03:07:59.663673 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:59.650472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40135 (local address 127.5.122.62:42321)
0807 03:07:59.651281 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:59.651296 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:59.651351 (+ 55us) server_negotiation.cc:407] Connection header received
0807 03:07:59.654059 (+ 2708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:59.654087 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:59.654587 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:59.654988 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:59.655497 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:59.657460 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:59.658772 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:59.659789 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:59.659981 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:59.661177 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:59.661193 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:59.661214 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:59.661812 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:59.661889 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:59.661920 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:59.662080 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:59.662282 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 03:07:59.662467 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":328,"threads_started":1}
I20260807 03:07:59.663885 5697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e: No bootstrap required, opened a new log
I20260807 03:07:59.667156 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:07:59.671550 5697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42321 } }
I20260807 03:07:59.671958 5697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:07:59.672053 5697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a9a09ce381343a2a3c53f2dcdde169e, State: Initialized, Role: FOLLOWER
I20260807 03:07:59.672685 5697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42321 } }
I20260807 03:07:59.672945 5697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:07:59.673080 5697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:07:59.673238 5697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:07:59.676156 5697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42321 } }
I20260807 03:07:59.676713 5697 leader_election.cc:302] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a9a09ce381343a2a3c53f2dcdde169e; no voters:
I20260807 03:07:59.677398 5697 leader_election.cc:288] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:07:59.677505 5712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:07:59.677835 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:07:59.671091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47681 (local address 127.5.122.62:42321)
0807 03:07:59.671219 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 03:07:59.671223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:07:59.671757 (+ 534us) server_negotiation.cc:407] Connection header received
0807 03:07:59.671926 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:07:59.671932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:07:59.672012 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:07:59.672103 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:07:59.672684 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:59.673558 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:59.675367 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:07:59.675795 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:07:59.675885 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:07:59.677002 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:07:59.677007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:07:59.677010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:07:59.677026 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:07:59.677079 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:07:59.677095 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:07:59.677193 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:07:59.677421 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 03:07:59.677483 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 03:07:59.679800 5697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:07:59.682545 5712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [term 1 LEADER]: Becoming Leader. State: Replica: 0a9a09ce381343a2a3c53f2dcdde169e, State: Running, Role: LEADER
I20260807 03:07:59.683171 5712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42321 } }
I20260807 03:07:59.686127 5718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a9a09ce381343a2a3c53f2dcdde169e. Latest consensus state: current_term: 1 leader_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42321 } } }
I20260807 03:07:59.686580 5717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9a09ce381343a2a3c53f2dcdde169e" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 42321 } } }
I20260807 03:07:59.686894 5718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [sys.catalog]: This master's current role is: LEADER
I20260807 03:07:59.686892 5717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e [sys.catalog]: This master's current role is: LEADER
W20260807 03:07:59.690236 5742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 03:07:59.690352 5742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:07:59.690479 5744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:07:59.692022 5744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:07:59.699747 5744 catalog_manager.cc:1347] Generated new cluster ID: 6d83aae6a7ef47a5a0250c5908709199
I20260807 03:07:59.699846 5744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:07:59.714138 5744 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:07:59.715590 5744 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:07:59.721683 5744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a9a09ce381343a2a3c53f2dcdde169e: Generated new TSK 0
I20260807 03:07:59.722776 5744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:07:59.728632 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5582
W20260807 03:07:59.729616 5688 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40135
W20260807 03:07:59.730273 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5582: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:07:59.730455 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5582
W20260807 03:07:59.752978 5556 connection.cc:535] client connection to 127.5.122.62:42321 recv error: Network error: failed to read from TLS socket (remote: 127.5.122.62:42321): Connection reset by peer (error 104)
2026-08-07T03:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:07:59 dist-test-slave-fgb3 krb5kdc[5752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:07:59 dist-test-slave-fgb3 krb5kdc[5752](info): set up 2 sockets
Aug 07 03:07:59 dist-test-slave-fgb3 krb5kdc[5752](info): commencing operation
krb5kdc: starting...
W20260807 03:08:00.923715 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:08:01 dist-test-slave-fgb3 krb5kdc[5752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:08:01.219780 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:38363
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:38363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:08:01.540501 5774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:01.540798 5774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:01.540938 5774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:08:01.548062 5774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:01.548172 5774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:01.548246 5774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:01.548327 5774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:01.548386 5774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:01.558554 5774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:38363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:38363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:01.559911 5774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:01.561232 5774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:01.572329 5783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:01.572261 5784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:01.572285 5787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:01.572896 5774 server_base.cc:1034] running on GCE node
I20260807 03:08:01.573738 5774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:01.575093 5774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:01.576296 5774 hybrid_clock.cc:648] HybridClock initialized: now 1786072081576250 us; error 35 us; skew 500 ppm
I20260807 03:08:01.576630 5774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:01 dist-test-slave-fgb3 krb5kdc[5752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072081, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:01.583753 5774 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:08:01.585179 5774 webserver.cc:460] Webserver started at http://127.5.122.62:41435/ using document root <none> and password file <none>
I20260807 03:08:01.585812 5774 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:01.585913 5774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:01.586199 5774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:01.589094 5774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "e68152160aa24b24a5cc38265f1b7fcb"
format_stamp: "Formatted at 2026-08-07 03:08:01 on dist-test-slave-fgb3"
server_key: "46f3d38b7b032b3ecaf1ed4808f68191"
server_key_iv: "7d28f0c285ff89071b0c58272695f79a"
server_key_version: "encryptionkey@0"
I20260807 03:08:01.589793 5774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "e68152160aa24b24a5cc38265f1b7fcb"
format_stamp: "Formatted at 2026-08-07 03:08:01 on dist-test-slave-fgb3"
server_key: "46f3d38b7b032b3ecaf1ed4808f68191"
server_key_iv: "7d28f0c285ff89071b0c58272695f79a"
server_key_version: "encryptionkey@0"
I20260807 03:08:01.594625 5774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260807 03:08:01.597748 5799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:01.598973 5774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:08:01.599126 5774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "e68152160aa24b24a5cc38265f1b7fcb"
format_stamp: "Formatted at 2026-08-07 03:08:01 on dist-test-slave-fgb3"
server_key: "46f3d38b7b032b3ecaf1ed4808f68191"
server_key_iv: "7d28f0c285ff89071b0c58272695f79a"
server_key_version: "encryptionkey@0"
I20260807 03:08:01.599291 5774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:01.604665 5774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:01.608261 5774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:01.608515 5774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:01.624974 5774 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:38363
I20260807 03:08:01.625015 5868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:38363 every 8 connection(s)
I20260807 03:08:01.626204 5774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:08:01.629603 5870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:01.636477 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5774
I20260807 03:08:01.636648 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:08:01.637202 5608 external_mini_cluster.cc:1428] Setting key 6cd9f9a151290114e0dbc76222dcabbb
I20260807 03:08:01.640214 5870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb: Bootstrap starting.
I20260807 03:08:01.643235 5870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:01.644304 5870 log.cc:824] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:01.644456 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:01.638925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35901 (local address 127.5.122.62:38363)
0807 03:08:01.639704 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:01.639715 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:01.639753 (+ 38us) server_negotiation.cc:407] Connection header received
0807 03:08:01.641404 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:01.641429 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:01.641739 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:01.642073 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:01.643055 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:01.643062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:01.643073 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:08:01.643087 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:01.643145 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:08:01.643164 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:01.643325 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:01.643456 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 03:08:01.643637 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":337,"threads_started":1}
I20260807 03:08:01.647012 5870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb: No bootstrap required, opened a new log
I20260807 03:08:01.647759 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:08:01.653949 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:01.651683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37493 (local address 127.5.122.62:38363)
0807 03:08:01.651850 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:01.651856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:01.652309 (+ 453us) server_negotiation.cc:407] Connection header received
0807 03:08:01.652460 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:01.652463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:01.652504 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:01.652578 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:01.653413 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:01.653417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:01.653419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:08:01.653432 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:01.653476 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:08:01.653490 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:01.653578 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:01.653666 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 03:08:01.653701 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:08:01.654901 5870 raft_consensus.cc:348] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68152160aa24b24a5cc38265f1b7fcb" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38363 } }
I20260807 03:08:01.655212 5870 raft_consensus.cc:374] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:08:01.655309 5870 raft_consensus.cc:729] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68152160aa24b24a5cc38265f1b7fcb, State: Initialized, Role: FOLLOWER
I20260807 03:08:01.655791 5870 consensus_queue.cc:260] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68152160aa24b24a5cc38265f1b7fcb" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38363 } }
I20260807 03:08:01.656005 5870 raft_consensus.cc:388] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:08:01.656133 5870 raft_consensus.cc:482] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:08:01.656286 5870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:01.658286 5870 raft_consensus.cc:504] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68152160aa24b24a5cc38265f1b7fcb" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38363 } }
I20260807 03:08:01.658804 5870 leader_election.cc:302] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e68152160aa24b24a5cc38265f1b7fcb; no voters:
I20260807 03:08:01.659333 5870 leader_election.cc:288] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:08:01.659451 5885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:08:01.660030 5885 raft_consensus.cc:686] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [term 1 LEADER]: Becoming Leader. State: Replica: e68152160aa24b24a5cc38265f1b7fcb, State: Running, Role: LEADER
I20260807 03:08:01.660483 5885 consensus_queue.cc:237] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68152160aa24b24a5cc38265f1b7fcb" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38363 } }
I20260807 03:08:01.661075 5870 sys_catalog.cc:564] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:01.662900 5887 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e68152160aa24b24a5cc38265f1b7fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68152160aa24b24a5cc38265f1b7fcb" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38363 } } }
I20260807 03:08:01.663293 5887 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:01.664752 5895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:08:01.666795 5889 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e68152160aa24b24a5cc38265f1b7fcb. Latest consensus state: current_term: 1 leader_uuid: "e68152160aa24b24a5cc38265f1b7fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68152160aa24b24a5cc38265f1b7fcb" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 38363 } } }
I20260807 03:08:01.667074 5889 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:01.675590 5895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:08:01.683604 5895 catalog_manager.cc:1347] Generated new cluster ID: b28867c0192a4c11a3cfa1c3869019ae
I20260807 03:08:01.683696 5895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:08:01.707581 5895 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:08:01.707942 5895 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:08:01.719312 5895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e68152160aa24b24a5cc38265f1b7fcb: Generated new TSK 0
I20260807 03:08:01.719996 5895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:08:01.728431 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5774
W20260807 03:08:01.729578 5864 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35901
W20260807 03:08:01.730257 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5774: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:08:01.730427 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5774
2026-08-07T03:08:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1997 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:08:01 dist-test-slave-fgb3 krb5kdc[5917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:08:01 dist-test-slave-fgb3 krb5kdc[5917](info): set up 2 sockets
Aug 07 03:08:01 dist-test-slave-fgb3 krb5kdc[5917](info): commencing operation
krb5kdc: starting...
W20260807 03:08:02.930562 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:08:03 dist-test-slave-fgb3 krb5kdc[5917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:08:03.225201 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45705
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:45705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:08:03.548633 5935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:03.548977 5935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:03.549139 5935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:08:03.556582 5935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:03.556732 5935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:03.556823 5935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:03.556915 5935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:03.556981 5935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:03.567530 5935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:45705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:03.568809 5935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:03.570106 5935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:03.581545 5945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:03.581584 5946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:03.581741 5949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:03.582404 5935 server_base.cc:1034] running on GCE node
I20260807 03:08:03.583065 5935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:03.584281 5935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:03.585477 5935 hybrid_clock.cc:648] HybridClock initialized: now 1786072083585435 us; error 31 us; skew 500 ppm
I20260807 03:08:03.585793 5935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:03 dist-test-slave-fgb3 krb5kdc[5917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072083, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:03.588727 5935 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:08:03.590030 5935 webserver.cc:460] Webserver started at http://127.5.122.62:37609/ using document root <none> and password file <none>
I20260807 03:08:03.590714 5935 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:03.590833 5935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:03.591113 5935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:03.593902 5935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "640bfdbbc0b2417ca6625507fe1a0ae5"
format_stamp: "Formatted at 2026-08-07 03:08:03 on dist-test-slave-fgb3"
server_key: "9d573b3164e775a0c3c580e7b74bf00f"
server_key_iv: "b2a7f54e22347f4c111ebb5ab6973bb2"
server_key_version: "encryptionkey@0"
I20260807 03:08:03.594652 5935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "640bfdbbc0b2417ca6625507fe1a0ae5"
format_stamp: "Formatted at 2026-08-07 03:08:03 on dist-test-slave-fgb3"
server_key: "9d573b3164e775a0c3c580e7b74bf00f"
server_key_iv: "b2a7f54e22347f4c111ebb5ab6973bb2"
server_key_version: "encryptionkey@0"
I20260807 03:08:03.599305 5935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 03:08:03.602420 5957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:03.603634 5935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 03:08:03.603783 5935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "640bfdbbc0b2417ca6625507fe1a0ae5"
format_stamp: "Formatted at 2026-08-07 03:08:03 on dist-test-slave-fgb3"
server_key: "9d573b3164e775a0c3c580e7b74bf00f"
server_key_iv: "b2a7f54e22347f4c111ebb5ab6973bb2"
server_key_version: "encryptionkey@0"
I20260807 03:08:03.603951 5935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:03.615629 5935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:03.619421 5935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:03.619669 5935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:03.635633 5935 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:45705
I20260807 03:08:03.635661 6029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:45705 every 8 connection(s)
I20260807 03:08:03.636878 5935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:08:03.640412 6030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:03.643225 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 5935
I20260807 03:08:03.643373 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:08:03.643785 5608 external_mini_cluster.cc:1428] Setting key b77d111b4ecd5f8ae9efaacd9d61da25
I20260807 03:08:03.652616 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5: Bootstrap starting.
I20260807 03:08:03.656416 6030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:03.657686 6030 log.cc:824] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5: Log is configured to *not* fsync() on all Append() calls
Aug 07 03:08:03 dist-test-slave-fgb3 krb5kdc[5917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:03.661410 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5: No bootstrap required, opened a new log
I20260807 03:08:03.668084 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:03.645480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49717 (local address 127.5.122.62:45705)
0807 03:08:03.646252 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:03.646266 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:03.646333 (+ 67us) server_negotiation.cc:407] Connection header received
0807 03:08:03.648560 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:03.648586 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:03.649100 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:03.649514 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:03.650813 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:03.652727 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:03.654494 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:03.655515 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:03.655684 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:03.660539 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:03.660547 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:03.660563 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:03.660605 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:03.664123 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:03.664790 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:03.664797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:03.664806 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:03.664975 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:03.665339 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:03.665343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:03.665345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:03.665881 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:03.666185 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:03.666634 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 03:08:03.667069 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":323,"threads_started":1}
I20260807 03:08:03.669618 6030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45705 } }
I20260807 03:08:03.669979 6030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:08:03.670079 6030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640bfdbbc0b2417ca6625507fe1a0ae5, State: Initialized, Role: FOLLOWER
I20260807 03:08:03.670682 6030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45705 } }
I20260807 03:08:03.670951 6030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:08:03.671085 6030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:08:03.671241 6030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:03.671608 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:08:03.673414 6030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45705 } }
I20260807 03:08:03.673870 6030 leader_election.cc:302] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 640bfdbbc0b2417ca6625507fe1a0ae5; no voters:
I20260807 03:08:03.674499 6030 leader_election.cc:288] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:08:03.674631 6039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:08:03.675792 6039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [term 1 LEADER]: Becoming Leader. State: Replica: 640bfdbbc0b2417ca6625507fe1a0ae5, State: Running, Role: LEADER
I20260807 03:08:03.676389 6039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45705 } }
I20260807 03:08:03.677146 6030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:03.678915 6046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 640bfdbbc0b2417ca6625507fe1a0ae5. Latest consensus state: current_term: 1 leader_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45705 } } }
I20260807 03:08:03.679292 6046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:03.679863 6045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bfdbbc0b2417ca6625507fe1a0ae5" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45705 } } }
I20260807 03:08:03.680169 6045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:03.693975 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:03.676330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55309 (local address 127.5.122.62:45705)
0807 03:08:03.677000 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:03.677006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:03.677022 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:08:03.677069 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:03.677074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:03.677152 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:03.677254 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:03.679365 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:03.680503 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:03.682496 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:03.683192 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:03.683298 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:03.686499 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:03.686506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:03.686511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:03.686552 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:03.690044 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:03.690890 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:03.690896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:03.690900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:03.691065 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:03.691496 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:03.691501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:03.691505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:03.691979 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:03.692230 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:03.693366 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0807 03:08:03.693595 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558}
I20260807 03:08:03.701052 6050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:08:03.706754 6050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:08:03.714751 6050 catalog_manager.cc:1347] Generated new cluster ID: ddb4a5e3ab734030b74dbd9c217b688c
I20260807 03:08:03.714855 6050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:08:03.735496 6050 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:08:03.736761 6050 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:08:03.742244 6050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 640bfdbbc0b2417ca6625507fe1a0ae5: Generated new TSK 0
I20260807 03:08:03.743026 6050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:08:03.760732 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 5935
I20260807 03:08:03.761822 6025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:08:03.850673 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 5935
2026-08-07T03:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2120 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:08:03 dist-test-slave-fgb3 krb5kdc[6073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:08:03 dist-test-slave-fgb3 krb5kdc[6073](info): set up 2 sockets
Aug 07 03:08:03 dist-test-slave-fgb3 krb5kdc[6073](info): commencing operation
krb5kdc: starting...
W20260807 03:08:05.044234 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:08:05 dist-test-slave-fgb3 krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:08:05.336716 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40443
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:40443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:08:05.659484 6088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:05.659739 6088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:05.659834 6088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:08:05.666778 6088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:05.666880 6088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:05.666956 6088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:05.667033 6088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:05.667090 6088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:05.676440 6088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:40443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:40443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:05.677592 6088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:05.678840 6088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:05.689559 6094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:05.689896 6099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:05.690001 6097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:05.690608 6088 server_base.cc:1034] running on GCE node
I20260807 03:08:05.691269 6088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:05.692508 6088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:05.693707 6088 hybrid_clock.cc:648] HybridClock initialized: now 1786072085693663 us; error 33 us; skew 500 ppm
I20260807 03:08:05.694031 6088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:05 dist-test-slave-fgb3 krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072085, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:05.696965 6088 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:08:05.698293 6088 webserver.cc:460] Webserver started at http://127.5.122.62:45125/ using document root <none> and password file <none>
I20260807 03:08:05.698936 6088 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:05.699034 6088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:05.699330 6088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:05.702072 6088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "37ffb19fa10844c297d5199522e35429"
format_stamp: "Formatted at 2026-08-07 03:08:05 on dist-test-slave-fgb3"
server_key: "f4290a7590ba7f2cefa5b8b088150116"
server_key_iv: "48b5cc728c1ee54d2e28ca872d9f9e34"
server_key_version: "encryptionkey@0"
I20260807 03:08:05.702803 6088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "37ffb19fa10844c297d5199522e35429"
format_stamp: "Formatted at 2026-08-07 03:08:05 on dist-test-slave-fgb3"
server_key: "f4290a7590ba7f2cefa5b8b088150116"
server_key_iv: "48b5cc728c1ee54d2e28ca872d9f9e34"
server_key_version: "encryptionkey@0"
I20260807 03:08:05.707480 6088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 03:08:05.710620 6107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:05.711817 6088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 03:08:05.711971 6088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "37ffb19fa10844c297d5199522e35429"
format_stamp: "Formatted at 2026-08-07 03:08:05 on dist-test-slave-fgb3"
server_key: "f4290a7590ba7f2cefa5b8b088150116"
server_key_iv: "48b5cc728c1ee54d2e28ca872d9f9e34"
server_key_version: "encryptionkey@0"
I20260807 03:08:05.712141 6088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:05.750144 6088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:05.754021 6088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:05.754289 6088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:05.771167 6088 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:40443
I20260807 03:08:05.771234 6179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:40443 every 8 connection(s)
I20260807 03:08:05.772452 6088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:08:05.775981 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:05.781620 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6088
I20260807 03:08:05.781765 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:08:05.782173 5608 external_mini_cluster.cc:1428] Setting key de03205fba905506c58f929aa23f2b3c
I20260807 03:08:05.786158 6180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429: Bootstrap starting.
I20260807 03:08:05.789363 6180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:05.790572 6180 log.cc:824] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:05.793411 6180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429: No bootstrap required, opened a new log
I20260807 03:08:05.794751 6185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:05.783815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51799 (local address 127.5.122.62:40443)
0807 03:08:05.784596 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:05.784607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:05.784646 (+ 39us) server_negotiation.cc:407] Connection header received
0807 03:08:05.786212 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:05.786231 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:05.786630 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:05.786929 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:05.787541 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:05.788819 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:05.790575 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:05.791445 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:05.791633 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:05.792684 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:05.792691 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:05.792703 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:08:05.793183 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:05.793255 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:08:05.793278 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:05.793362 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:05.793597 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 03:08:05.793815 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":352,"threads_started":1}
I20260807 03:08:05.798060 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:08:05.802192 6180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ffb19fa10844c297d5199522e35429" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40443 } }
I20260807 03:08:05.802634 6180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:08:05.802740 6180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ffb19fa10844c297d5199522e35429, State: Initialized, Role: FOLLOWER
I20260807 03:08:05.803287 6180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ffb19fa10844c297d5199522e35429" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40443 } }
I20260807 03:08:05.803508 6180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:08:05.803630 6180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:08:05.803779 6180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:05.805764 6180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ffb19fa10844c297d5199522e35429" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40443 } }
I20260807 03:08:05.806257 6180 leader_election.cc:302] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37ffb19fa10844c297d5199522e35429; no voters:
I20260807 03:08:05.807004 6180 leader_election.cc:288] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:08:05.807111 6194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:08:05.808174 6194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [term 1 LEADER]: Becoming Leader. State: Replica: 37ffb19fa10844c297d5199522e35429, State: Running, Role: LEADER
I20260807 03:08:05.808773 6194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ffb19fa10844c297d5199522e35429" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40443 } }
I20260807 03:08:05.809144 6180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:05.811478 6196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37ffb19fa10844c297d5199522e35429. Latest consensus state: current_term: 1 leader_uuid: "37ffb19fa10844c297d5199522e35429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ffb19fa10844c297d5199522e35429" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40443 } } }
I20260807 03:08:05.811867 6196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:05.812376 6195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37ffb19fa10844c297d5199522e35429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ffb19fa10844c297d5199522e35429" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 40443 } } }
I20260807 03:08:05.812620 6195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:05.813046 6185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:05.802890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44337 (local address 127.5.122.62:40443)
0807 03:08:05.803044 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:05.803049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:05.806470 (+ 3421us) server_negotiation.cc:407] Connection header received
0807 03:08:05.806629 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:05.806633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:05.806686 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:05.806771 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:05.807319 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:05.808364 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:05.810035 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:05.810737 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:05.810854 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:05.812005 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:05.812011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:05.812016 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:08:05.812038 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:05.812085 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:08:05.812106 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:05.812204 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:05.812599 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 03:08:05.812695 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 03:08:05.813695 6201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:08:05.819680 6201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:08:05.830020 6201 catalog_manager.cc:1347] Generated new cluster ID: 13fcb23ad1d544dd8b2b11055b87a4c5
I20260807 03:08:05.830154 6201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:08:05.847255 6201 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:08:05.848562 6201 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:08:05.855052 6201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37ffb19fa10844c297d5199522e35429: Generated new TSK 0
I20260807 03:08:05.855836 6201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:08:05.864643 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6088
W20260807 03:08:05.865650 6171 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51799
W20260807 03:08:05.866333 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6088: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:08:05.866492 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6088
2026-08-07T03:08:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:08:05 dist-test-slave-fgb3 krb5kdc[6216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:08:05 dist-test-slave-fgb3 krb5kdc[6216](info): set up 2 sockets
Aug 07 03:08:05 dist-test-slave-fgb3 krb5kdc[6216](info): commencing operation
krb5kdc: starting...
W20260807 03:08:07.060815 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:08:07 dist-test-slave-fgb3 krb5kdc[6216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:08:07.344916 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45599
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:45571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:45599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:08:07.662961 6232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:07.663213 6232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:07.663309 6232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:08:07.670176 6232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:07.670266 6232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:07.670334 6232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:07.670408 6232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:07.670454 6232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:07.679675 6232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:45571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:45599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:45599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:07.680745 6232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:07.681886 6232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:07.692377 6241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:07.692412 6245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:07.692348 6242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:07.693423 6232 server_base.cc:1034] running on GCE node
I20260807 03:08:07.694064 6232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:07.695324 6232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:07.696507 6232 hybrid_clock.cc:648] HybridClock initialized: now 1786072087696468 us; error 46 us; skew 500 ppm
I20260807 03:08:07.696817 6232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:07 dist-test-slave-fgb3 krb5kdc[6216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072087, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:07.703289 6232 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:08:07.704695 6232 webserver.cc:460] Webserver started at http://127.5.122.62:42443/ using document root <none> and password file <none>
I20260807 03:08:07.705348 6232 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:07.705462 6232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:07.705739 6232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:07.708609 6232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "240eff1664d74c9eae37beaff3bf7114"
format_stamp: "Formatted at 2026-08-07 03:08:07 on dist-test-slave-fgb3"
server_key: "8af08f6a03be8954e9090a04fc5d1d94"
server_key_iv: "d3b9b00080899b48fe04cfc50d03f51f"
server_key_version: "encryptionkey@0"
I20260807 03:08:07.709318 6232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "240eff1664d74c9eae37beaff3bf7114"
format_stamp: "Formatted at 2026-08-07 03:08:07 on dist-test-slave-fgb3"
server_key: "8af08f6a03be8954e9090a04fc5d1d94"
server_key_iv: "d3b9b00080899b48fe04cfc50d03f51f"
server_key_version: "encryptionkey@0"
I20260807 03:08:07.714102 6232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260807 03:08:07.717252 6253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:07.718461 6232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 03:08:07.718609 6232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "240eff1664d74c9eae37beaff3bf7114"
format_stamp: "Formatted at 2026-08-07 03:08:07 on dist-test-slave-fgb3"
server_key: "8af08f6a03be8954e9090a04fc5d1d94"
server_key_iv: "d3b9b00080899b48fe04cfc50d03f51f"
server_key_version: "encryptionkey@0"
I20260807 03:08:07.718778 6232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:07.724088 6232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:07.727871 6232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:07.728128 6232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:07.744491 6232 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:45599
I20260807 03:08:07.744549 6317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:45599 every 8 connection(s)
I20260807 03:08:07.745810 6232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:08:07.746717 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6232
I20260807 03:08:07.746896 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:08:07.747413 5608 external_mini_cluster.cc:1428] Setting key a0daa5402994a37ec323202ed67737be
I20260807 03:08:07.749953 6318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:07.754621 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:07.749117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40927 (local address 127.5.122.62:45599)
0807 03:08:07.749859 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:07.749870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:07.749912 (+ 42us) server_negotiation.cc:407] Connection header received
0807 03:08:07.751528 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:07.751548 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:07.751839 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:07.752148 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:07.753133 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:07.753139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:07.753150 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:08:07.753164 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:07.753227 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:08:07.753248 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:07.753432 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:07.753522 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 03:08:07.753669 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":328,"threads_started":1}
I20260807 03:08:07.757925 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:08:07.761745 6318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114: Bootstrap starting.
I20260807 03:08:07.765239 6318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:07.765869 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:07.763598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50639 (local address 127.5.122.62:45599)
0807 03:08:07.763772 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:07.763778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:07.764148 (+ 370us) server_negotiation.cc:407] Connection header received
0807 03:08:07.764275 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:07.764279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:07.764337 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:07.764436 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:07.765255 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:07.765261 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:07.765265 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 03:08:07.765283 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:07.765330 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 03:08:07.765350 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:07.765485 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:07.765534 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 03:08:07.765598 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:08:07.766491 6318 log.cc:824] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:07.768921 6318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114: No bootstrap required, opened a new log
I20260807 03:08:07.775739 6318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "240eff1664d74c9eae37beaff3bf7114" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45599 } }
I20260807 03:08:07.776077 6318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 03:08:07.776160 6318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 240eff1664d74c9eae37beaff3bf7114, State: Initialized, Role: FOLLOWER
I20260807 03:08:07.776666 6318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "240eff1664d74c9eae37beaff3bf7114" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45599 } }
I20260807 03:08:07.776877 6318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 03:08:07.777004 6318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 03:08:07.777153 6318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:07.779333 6318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "240eff1664d74c9eae37beaff3bf7114" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45599 } }
I20260807 03:08:07.779770 6318 leader_election.cc:302] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 240eff1664d74c9eae37beaff3bf7114; no voters:
I20260807 03:08:07.780310 6318 leader_election.cc:288] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 03:08:07.780442 6330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:08:07.781047 6330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [term 1 LEADER]: Becoming Leader. State: Replica: 240eff1664d74c9eae37beaff3bf7114, State: Running, Role: LEADER
I20260807 03:08:07.781605 6330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "240eff1664d74c9eae37beaff3bf7114" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45599 } }
I20260807 03:08:07.782346 6318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:07.784284 6331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "240eff1664d74c9eae37beaff3bf7114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "240eff1664d74c9eae37beaff3bf7114" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45599 } } }
I20260807 03:08:07.784749 6331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:07.784359 6332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 240eff1664d74c9eae37beaff3bf7114. Latest consensus state: current_term: 1 leader_uuid: "240eff1664d74c9eae37beaff3bf7114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "240eff1664d74c9eae37beaff3bf7114" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 45599 } } }
I20260807 03:08:07.785492 6332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:07.786072 6339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:08:07.793706 6339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:08:07.801836 6339 catalog_manager.cc:1347] Generated new cluster ID: a5e384238b2e45cd872b6efaa465625a
I20260807 03:08:07.801934 6339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:08:07.819761 6339 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:08:07.820194 6339 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:08:07.828562 6339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 240eff1664d74c9eae37beaff3bf7114: Generated new TSK 0
I20260807 03:08:07.829336 6339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:08:07.836354 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6232
W20260807 03:08:07.837515 6313 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40927
W20260807 03:08:07.838171 5608 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6232: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 03:08:07.838369 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6232
2026-08-07T03:08:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1972 ms)
[----------] 18 tests from AuthTokenIssuingTest (37783 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:08:07 dist-test-slave-fgb3 krb5kdc[6357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:08:07 dist-test-slave-fgb3 krb5kdc[6357](info): set up 2 sockets
Aug 07 03:08:07 dist-test-slave-fgb3 krb5kdc[6357](info): commencing operation
krb5kdc: starting...
W20260807 03:08:09.035547 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.007s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:08:09 dist-test-slave-fgb3 krb5kdc[6357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:08:09.329069 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:37081
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:37081,127.5.122.61:43755,127.5.122.60:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:08:09.656962 6378 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 03:08:09.657248 6378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:09.657346 6378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:09.657462 6378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:08:09.664295 6378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:09.664376 6378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:09.664471 6378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:09.664538 6378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:09.664623 6378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:09.673792 6378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:37081,127.5.122.61:43755,127.5.122.60:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:37081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:09.674985 6378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:09.676198 6378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:09.687147 6383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:09.687237 6384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:09.687270 6388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:09.688189 6378 server_base.cc:1034] running on GCE node
I20260807 03:08:09.688916 6378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:09.690186 6378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:09.691388 6378 hybrid_clock.cc:648] HybridClock initialized: now 1786072089691341 us; error 44 us; skew 500 ppm
I20260807 03:08:09.691691 6378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:09 dist-test-slave-fgb3 krb5kdc[6357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:09.694586 6378 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:08:09.695966 6378 webserver.cc:460] Webserver started at http://127.5.122.62:39019/ using document root <none> and password file <none>
I20260807 03:08:09.696609 6378 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:09.696702 6378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:09.697029 6378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:09.699954 6378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "ae92616e623f43e0a0da8f6bfbfc2179"
format_stamp: "Formatted at 2026-08-07 03:08:09 on dist-test-slave-fgb3"
server_key: "fc58592ab48c2900bf0c496a73c4817e"
server_key_iv: "7b818cdd0ef9035b338a4eb8cd2a021e"
server_key_version: "encryptionkey@0"
I20260807 03:08:09.700695 6378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "ae92616e623f43e0a0da8f6bfbfc2179"
format_stamp: "Formatted at 2026-08-07 03:08:09 on dist-test-slave-fgb3"
server_key: "fc58592ab48c2900bf0c496a73c4817e"
server_key_iv: "7b818cdd0ef9035b338a4eb8cd2a021e"
server_key_version: "encryptionkey@0"
I20260807 03:08:09.705544 6378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 03:08:09.708803 6399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:09.710095 6378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:08:09.710254 6378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "ae92616e623f43e0a0da8f6bfbfc2179"
format_stamp: "Formatted at 2026-08-07 03:08:09 on dist-test-slave-fgb3"
server_key: "fc58592ab48c2900bf0c496a73c4817e"
server_key_iv: "7b818cdd0ef9035b338a4eb8cd2a021e"
server_key_version: "encryptionkey@0"
I20260807 03:08:09.710525 6378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:09.727699 6378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:09.731528 6378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:09.731782 6378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:09.748786 6378 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:37081
I20260807 03:08:09.748822 6474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:37081 every 8 connection(s)
I20260807 03:08:09.750121 6378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:08:09.753710 6475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:09.757247 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6378
I20260807 03:08:09.757472 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:08:09.758291 5608 external_mini_cluster.cc:1428] Setting key d67273009ea6032a9526634059eeab54
I20260807 03:08:09.761441 6475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:09 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:09.784289 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.764888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37081 (local address 127.0.0.1:39808)
0807 03:08:09.765742 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:09.765789 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:09.768056 (+ 2267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:09.768820 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:09.768827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:09.769502 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:09.769931 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:09.769953 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:09.771470 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:09.771475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:09.772943 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:09.772955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:09.774091 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:09.774095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:09.774211 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:09.774836 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:09.774857 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:09.777070 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:09.780666 (+ 3596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:09.780676 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:09.780685 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:09.781050 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:09.781467 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:09.781470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:09.781473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:09.781615 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:09.782512 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:09.782518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:09.782706 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 03:08:09.782907 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 03:08:09.783276 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":363,"threads_started":1}
I20260807 03:08:09.784314 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.764888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.5.122.62:37081)
0807 03:08:09.766650 (+ 1762us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:09.766656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:09.766701 (+ 45us) server_negotiation.cc:407] Connection header received
0807 03:08:09.768392 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:09.768397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:09.768603 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:09.768752 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:09.770135 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:09.771325 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:09.773075 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:09.773889 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:09.774068 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:09.777293 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:09.777303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:09.777317 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:09.777355 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:09.780533 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:09.781172 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:09.781181 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:09.781184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:09.781364 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:09.781706 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:09.781711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:09.781713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:09.782382 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:09.782580 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:09.783063 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 03:08:09.783465 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":1572,"spinlock_wait_cycles":22272,"thread_start_us":1333,"threads_started":1}
I20260807 03:08:09.787045 6475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:09.788074 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.787736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:44982)
0807 03:08:09.787879 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:09.787977 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 03:08:09.788316 6404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
I20260807 03:08:09.789758 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.789451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:44998)
0807 03:08:09.789577 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:09.789637 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 03:08:09.789937 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:43755: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.122.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab.
Entry for principal kudu/127.5.122.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab.
I20260807 03:08:09.826589 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:43755
--webserver_interface=127.5.122.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab
--principal=kudu/127.5.122.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:37081,127.5.122.61:43755,127.5.122.60:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:08:09.839217 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } attempt: 1
I20260807 03:08:09.840271 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.839871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45006)
0807 03:08:09.840046 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:09.840129 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 03:08:09.841173 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.840836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45016)
0807 03:08:09.840983 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:09.841048 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 03:08:09.841339 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:43755: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
I20260807 03:08:09.909550 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } attempt: 2
I20260807 03:08:09.910625 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.910146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45018)
0807 03:08:09.910402 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:09.910488 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 03:08:09.911547 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:09.911208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45024)
0807 03:08:09.911342 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:09.911418 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 03:08:09.911726 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:43755: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
I20260807 03:08:10.002964 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } attempt: 3
I20260807 03:08:10.004132 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.003706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45030)
0807 03:08:10.003903 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.003994 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 03:08:10.005075 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.004721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45034)
0807 03:08:10.004864 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.004940 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 03:08:10.005242 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:43755: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
W20260807 03:08:10.146785 6486 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 03:08:10.147130 6486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:10.147228 6486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:10.147337 6486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 03:08:10.148447 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } attempt: 4
I20260807 03:08:10.149501 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.149009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45040)
0807 03:08:10.149287 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.149370 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 03:08:10.150513 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.150146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45050)
0807 03:08:10.150282 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.150391 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 03:08:10.150697 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:43755: Network error: Client connection negotiation failed: client connection to 127.5.122.61:43755: connect: Connection refused (error 111)
W20260807 03:08:10.154770 6486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:10.154855 6486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:10.154953 6486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:10.155017 6486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:10.155100 6486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:10.164741 6486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:37081,127.5.122.61:43755,127.5.122.60:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab
--principal=kudu/127.5.122.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:43755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:10.165952 6486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:10.167311 6486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:10.178911 6500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:10.178992 6496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:10.179016 6495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:10.179908 6486 server_base.cc:1034] running on GCE node
I20260807 03:08:10.180706 6486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:10.182183 6486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:10.183398 6486 hybrid_clock.cc:648] HybridClock initialized: now 1786072090183325 us; error 58 us; skew 500 ppm
I20260807 03:08:10.183698 6486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:10.186663 6486 init.cc:377] Logged in from keytab as kudu/127.5.122.61@KRBTEST.COM (short username kudu)
I20260807 03:08:10.187942 6486 webserver.cc:460] Webserver started at http://127.5.122.61:38085/ using document root <none> and password file <none>
I20260807 03:08:10.188567 6486 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:10.188663 6486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:10.188964 6486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:10.191783 6486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data/instance:
uuid: "3ae6d100de714201a12d52c2a904d4d5"
format_stamp: "Formatted at 2026-08-07 03:08:10 on dist-test-slave-fgb3"
server_key: "ca954fce5f2a78efd35a92f5eac1a9a0"
server_key_iv: "e40ae0a600dbe0f04a926d30d018d750"
server_key_version: "encryptionkey@0"
I20260807 03:08:10.192484 6486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/wal/instance:
uuid: "3ae6d100de714201a12d52c2a904d4d5"
format_stamp: "Formatted at 2026-08-07 03:08:10 on dist-test-slave-fgb3"
server_key: "ca954fce5f2a78efd35a92f5eac1a9a0"
server_key_iv: "e40ae0a600dbe0f04a926d30d018d750"
server_key_version: "encryptionkey@0"
I20260807 03:08:10.197068 6486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 03:08:10.200143 6509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:10.201299 6486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 03:08:10.201457 6486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/wal
uuid: "3ae6d100de714201a12d52c2a904d4d5"
format_stamp: "Formatted at 2026-08-07 03:08:10 on dist-test-slave-fgb3"
server_key: "ca954fce5f2a78efd35a92f5eac1a9a0"
server_key_iv: "e40ae0a600dbe0f04a926d30d018d750"
server_key_version: "encryptionkey@0"
I20260807 03:08:10.201639 6486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:10.230931 6486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:10.235131 6486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:10.235391 6486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:10.251884 6486 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.61:43755
I20260807 03:08:10.251904 6589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.61:43755 every 8 connection(s)
I20260807 03:08:10.253126 6486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
I20260807 03:08:10.256618 6591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:10.262898 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6486
I20260807 03:08:10.263055 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-1/wal/instance
I20260807 03:08:10.263445 5608 external_mini_cluster.cc:1428] Setting key e0bf65e4750052c5f970b8dfc0eb838a
I20260807 03:08:10.262869 6591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:10.281667 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.265304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.5.122.62:37081)
0807 03:08:10.265471 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.265476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.266786 (+ 1310us) server_negotiation.cc:407] Connection header received
0807 03:08:10.268560 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.268566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.268642 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.268731 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.270124 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.270834 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.272258 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.272660 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.272768 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.275954 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.275959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.275962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.275996 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.278972 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.279619 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.279625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.279627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.279763 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.280115 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.280120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.280122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.280445 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.280578 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.281141 (+ 563us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.281371 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 03:08:10.282337 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.265875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37081 (local address 127.0.0.1:39816)
0807 03:08:10.266630 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.266671 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.268295 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.268951 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.268959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.269430 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.269954 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.269970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.270985 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.270989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.272120 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.272131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.272864 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.272871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.273084 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.273675 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.273696 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.275749 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.279093 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.279099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.279109 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.279509 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.279854 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.279857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.279859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.280011 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.280550 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.280555 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.280813 (+ 258us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.281014 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.281338 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":344,"threads_started":1}
I20260807 03:08:10.283882 6591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.60@KRBTEST.COM; defaulting to no policy
I20260807 03:08:10.296662 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.284618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45058)
0807 03:08:10.284757 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.284773 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.284844 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.285810 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.285814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.286196 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.286354 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.286363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.287648 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.287653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.288457 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.288467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.289187 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.289192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.289347 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.289844 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.289858 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.291730 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.294602 (+ 2872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.294607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.294611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.294915 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.295279 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.295283 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.295286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.295429 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.295962 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.295965 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.296076 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.296179 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.296359 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 03:08:10.296862 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.284873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.5.122.61:43755)
0807 03:08:10.285463 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.285468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.285490 (+ 22us) server_negotiation.cc:407] Connection header received
0807 03:08:10.285526 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.285531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.285676 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.285801 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.286499 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.287526 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.288575 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.289063 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.289165 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.291913 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.291923 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.291934 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.291962 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.294497 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.295033 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.295039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.295041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.295166 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.295522 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.295525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.295527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.295864 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.295997 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.296300 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.296489 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":334,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.60@KRBTEST.COM" created.
I20260807 03:08:10.299041 6591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:10.299957 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.299571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:49952)
0807 03:08:10.299715 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.299837 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 03:08:10.300179 6516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.301538 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.301252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:49954)
0807 03:08:10.301372 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.301451 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 03:08:10.301725 6591 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab.
Entry for principal kudu/127.5.122.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab.
I20260807 03:08:10.329164 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:35243
--webserver_interface=127.5.122.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab
--principal=kudu/127.5.122.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:37081,127.5.122.61:43755,127.5.122.60:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:08:10.350986 6591 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 1
I20260807 03:08:10.352049 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.351616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:49970)
0807 03:08:10.351799 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.351909 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 03:08:10.352941 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.352639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:49986)
0807 03:08:10.352776 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.352844 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 03:08:10.353135 6591 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.421315 6591 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 2
I20260807 03:08:10.422390 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.421965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:49992)
0807 03:08:10.422133 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.422218 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 03:08:10.423298 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.423005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:49996)
0807 03:08:10.423129 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.423193 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 03:08:10.423503 6591 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.449947 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } attempt: 5
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:08:10.462980 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.450658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45066)
0807 03:08:10.450830 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.450851 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.450991 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.451341 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.451345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.451744 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.451970 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.451983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.452940 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.452953 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.453781 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.453801 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.454560 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.454566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.454764 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.455352 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.455369 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.457277 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.460847 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.460850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.460853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.461196 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.461543 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.461546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.461548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.461666 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.462239 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.462242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.462407 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.462550 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.462738 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 03:08:10.463210 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.450742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.5.122.61:43755)
0807 03:08:10.450926 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.450931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.450952 (+ 21us) server_negotiation.cc:407] Connection header received
0807 03:08:10.451116 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.451122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.451204 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.451284 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.452119 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.452813 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.453933 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.454411 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.454522 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.457469 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.457475 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.457480 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.457520 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.460722 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.461306 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.461312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.461315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.461456 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.461761 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.461765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.461769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.462138 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.462296 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.462681 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.462917 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 03:08:10.464200 6475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:10.465229 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.464778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50006)
0807 03:08:10.464951 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.465045 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 03:08:10.466182 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.465828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50008)
0807 03:08:10.465969 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.466064 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 03:08:10.466419 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.514689 6591 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 3
I20260807 03:08:10.515849 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.515390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50010)
0807 03:08:10.515605 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.515698 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 03:08:10.516963 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.516542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50020)
0807 03:08:10.516763 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.516840 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260807 03:08:10.517181 6591 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.517648 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 1
I20260807 03:08:10.518735 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.518212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50034)
0807 03:08:10.518427 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.518530 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 03:08:10.519654 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.519296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50050)
0807 03:08:10.519437 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.519520 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 03:08:10.519874 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.588079 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 2
I20260807 03:08:10.589295 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.588707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50052)
0807 03:08:10.588961 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.589085 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 03:08:10.590375 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.589980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50064)
0807 03:08:10.590131 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.590216 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 03:08:10.590607 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
W20260807 03:08:10.653172 6605 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 03:08:10.653489 6605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:10.653566 6605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:10.653645 6605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 03:08:10.660375 6591 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 4
W20260807 03:08:10.661034 6605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:10.661123 6605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:10.661248 6605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:10.661317 6605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:10.661410 6605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:10.661506 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.660990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50078)
0807 03:08:10.661222 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.661307 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 03:08:10.662452 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.662101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50080)
0807 03:08:10.662236 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.662306 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 03:08:10.662674 6591 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.671371 6605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:37081,127.5.122.61:43755,127.5.122.60:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab
--principal=kudu/127.5.122.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:10.672593 6605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:10.673859 6605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:10.684919 6615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:10.684994 6616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:10.685024 6618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:10.685770 6605 server_base.cc:1034] running on GCE node
I20260807 03:08:10.686573 6605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:10.687839 6605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:10.689056 6605 hybrid_clock.cc:648] HybridClock initialized: now 1786072090688999 us; error 36 us; skew 500 ppm
I20260807 03:08:10.689357 6605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:10.692256 6605 init.cc:377] Logged in from keytab as kudu/127.5.122.60@KRBTEST.COM (short username kudu)
I20260807 03:08:10.693552 6605 webserver.cc:460] Webserver started at http://127.5.122.60:40953/ using document root <none> and password file <none>
I20260807 03:08:10.694192 6605 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:10.694293 6605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:10.694617 6605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:10.696797 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 3
I20260807 03:08:10.697826 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.697392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50082)
0807 03:08:10.697591 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.697667 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 03:08:10.697577 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data/instance:
uuid: "e5f129a86f624ebb904a639e06a83723"
format_stamp: "Formatted at 2026-08-07 03:08:10 on dist-test-slave-fgb3"
server_key: "e8b191479032d43541e1a8afffa4e481"
server_key_iv: "77ce5c270953c2bb1dc2f90383cad71c"
server_key_version: "encryptionkey@0"
I20260807 03:08:10.698465 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/wal/instance:
uuid: "e5f129a86f624ebb904a639e06a83723"
format_stamp: "Formatted at 2026-08-07 03:08:10 on dist-test-slave-fgb3"
server_key: "e8b191479032d43541e1a8afffa4e481"
server_key_iv: "77ce5c270953c2bb1dc2f90383cad71c"
server_key_version: "encryptionkey@0"
I20260807 03:08:10.698798 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.698458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50090)
0807 03:08:10.698623 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.698695 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 03:08:10.699000 6475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:35243: Network error: Client connection negotiation failed: client connection to 127.5.122.60:35243: connect: Connection refused (error 111)
I20260807 03:08:10.703368 6605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 03:08:10.706578 6628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:10.707791 6605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 03:08:10.707942 6605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/wal
uuid: "e5f129a86f624ebb904a639e06a83723"
format_stamp: "Formatted at 2026-08-07 03:08:10 on dist-test-slave-fgb3"
server_key: "e8b191479032d43541e1a8afffa4e481"
server_key_iv: "77ce5c270953c2bb1dc2f90383cad71c"
server_key_version: "encryptionkey@0"
I20260807 03:08:10.708134 6605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:10.729823 6605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:10.733498 6605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:10.733737 6605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:10.751026 6605 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.60:35243
I20260807 03:08:10.751049 6693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.60:35243 every 8 connection(s)
I20260807 03:08:10.752264 6605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
I20260807 03:08:10.755890 6695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:10.762291 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6605
I20260807 03:08:10.762477 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786071733698599-5608-0/minicluster-data/master-2/wal/instance
I20260807 03:08:10.762132 6695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:10.762931 5608 external_mini_cluster.cc:1428] Setting key c29bbb6dba18fe1f6bcb8285d58eceab
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:08:10.782303 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.764862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39818 (local address 127.5.122.62:37081)
0807 03:08:10.765051 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.765056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.766071 (+ 1015us) server_negotiation.cc:407] Connection header received
0807 03:08:10.767737 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.767743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.767814 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.767917 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.769246 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.769965 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.771815 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.772304 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.772403 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.776296 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.776303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.776308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.776351 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.778997 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.779896 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.779899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.779901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.780023 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.780457 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.780462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.780466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.780803 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.780935 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.781729 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.781926 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:08:10.783603 6700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.765036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:37081 (local address 127.0.0.1:39818)
0807 03:08:10.765934 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.765973 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.767495 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.767998 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.768009 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.768725 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.769078 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.769094 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.770125 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.770131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.771664 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.771676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.772529 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.772537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.772769 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.773649 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.773686 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.776097 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.779125 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.779135 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.779148 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.779792 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.780117 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.780121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.780124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.780334 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.780923 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.780935 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.781320 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.781631 (+ 311us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.782142 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":51,"thread_start_us":432,"threads_started":1}
I20260807 03:08:10.783622 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.765036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.5.122.60:35243)
0807 03:08:10.765814 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.765823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.765859 (+ 36us) server_negotiation.cc:407] Connection header received
0807 03:08:10.767490 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.767511 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.767728 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.767886 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.769274 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.770689 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.771967 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.772606 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.772766 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.776820 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.776828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.776841 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.776875 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.780106 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.780867 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.780876 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.780879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.781032 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.781364 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.781369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.781372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.781786 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.782074 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.782419 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.782656 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":597,"thread_start_us":345,"threads_started":1}
I20260807 03:08:10.785153 6695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:10.798830 6700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.785927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:43755 (local address 127.0.0.1:45074)
0807 03:08:10.786162 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.786176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.786249 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.786877 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.786880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.787198 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.787342 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.787350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.788659 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.788665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.789635 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.789650 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.790354 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.790361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.790553 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.791134 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.791150 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.793433 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.796621 (+ 3188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.796625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.796628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.797036 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.797420 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.797424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.797426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.797555 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.798095 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.798098 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.798213 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.798337 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.798539 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260807 03:08:10.798980 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.785933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.5.122.61:43755)
0807 03:08:10.786471 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.786476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.786492 (+ 16us) server_negotiation.cc:407] Connection header received
0807 03:08:10.786541 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.786547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.786621 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.786720 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.787489 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.788525 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.789773 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.790198 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.790338 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.793612 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.793617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.793621 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.793653 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.796488 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.797149 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.797156 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.797159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.797321 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.797652 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.797657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.797661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.797976 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.798149 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.798455 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.798663 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420}
I20260807 03:08:10.799856 6695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:10.801626 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.786670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.5.122.62:37081)
0807 03:08:10.786992 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.786996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.787008 (+ 12us) server_negotiation.cc:407] Connection header received
0807 03:08:10.787044 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.787047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.787094 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.787168 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.788589 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.789319 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.790734 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.791244 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.791336 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.795441 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.795448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.795452 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.795489 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.798597 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.799430 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.799436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.799439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.799568 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.799923 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.799927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.799929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.800210 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.800468 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.801230 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.801386 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:08:10.812551 6700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.800508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50100)
0807 03:08:10.800651 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.800668 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.800741 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.800990 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.800993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.801386 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.801512 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.801520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.802840 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.802845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.803961 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.803972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.804877 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.804892 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.805108 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.805644 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.805666 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.807589 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.810172 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.810177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.810182 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.810581 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.810923 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.810928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.810931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.811080 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.811540 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.811545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.811714 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.811877 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.812151 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 03:08:10.812583 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.800576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.5.122.60:35243)
0807 03:08:10.800727 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.800732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.800747 (+ 15us) server_negotiation.cc:407] Connection header received
0807 03:08:10.800814 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.800819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.800885 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.800987 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.801643 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.802717 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.804088 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.804743 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.804874 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.807749 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.807753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.807756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.807782 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.810057 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.810683 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.810689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.810692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.810828 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.811171 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.811176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.811180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.811435 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.811559 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.811982 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.812304 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":56}
I20260807 03:08:10.817152 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.803224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.5.122.61:43755)
0807 03:08:10.803414 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.803420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.803437 (+ 17us) server_negotiation.cc:407] Connection header received
0807 03:08:10.803482 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.803486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.803550 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.803639 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.804938 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.805947 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.807169 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.807567 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.807671 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.811626 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.811633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.811638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.811672 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.814750 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.815386 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.815391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.815394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.815540 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.815843 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.815847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.815851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.816184 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.816451 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.816599 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.816821 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 03:08:10.818427 6695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: Bootstrap starting.
I20260807 03:08:10.821882 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:08:10.822088 6695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:10.823417 6695 log.cc:824] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:10.826113 6695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: No bootstrap required, opened a new log
I20260807 03:08:10.825198 6434 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ae92616e623f43e0a0da8f6bfbfc2179"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39828
I20260807 03:08:10.833205 6695 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:10.833530 6695 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f129a86f624ebb904a639e06a83723, State: Initialized, Role: FOLLOWER
I20260807 03:08:10.833948 6695 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:10.834873 6702 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:10.835278 6702 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:10.835780 6695 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 03:08:10.844769 6721 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 03:08:10.844918 6721 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:08:10.842824 6605 master_runner.cc:186] Error getting master registration for 127.5.122.62:37081: OK, instance_id { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" instance_seqno: 1786072089743853 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 03:08:10.853070 6605 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 03:08:10.876214 6475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 4
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072089, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:08:10.892872 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.877102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.5.122.60:35243)
0807 03:08:10.877290 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.877297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.877434 (+ 137us) server_negotiation.cc:407] Connection header received
0807 03:08:10.877630 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.877636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.877719 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.877828 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.878874 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.879991 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.881789 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.882574 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.882691 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.886456 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.886461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.886464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.886497 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.889942 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.890617 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.890623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.890627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.890770 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.891144 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.891149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.891153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.891644 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.891786 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.892247 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.892492 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:08:10.892882 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.877026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50116)
0807 03:08:10.877324 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.877346 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.877502 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.877889 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.877894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.878472 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.878704 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.878717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.880153 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.880159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.881653 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.881668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.882727 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.882734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.882932 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.883527 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.883551 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.886255 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.890071 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.890077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.890081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.890505 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.890879 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.890885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.890889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.891029 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.891789 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.891794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.891983 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.892138 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.892485 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260807 03:08:10.898805 6475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179: Bootstrap starting.
I20260807 03:08:10.902768 6475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:10.904034 6475 log.cc:824] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:10.907258 6475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179: No bootstrap required, opened a new log
I20260807 03:08:10.914660 6475 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:10.915055 6475 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae92616e623f43e0a0da8f6bfbfc2179, State: Initialized, Role: FOLLOWER
I20260807 03:08:10.915544 6475 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:10.916508 6725 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:10.916931 6725 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:10.917611 6475 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:10.917809 6434 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 03:08:10.918031 6434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:10.921341 6434 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:10.922824 6434 leader_election.cc:288] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [CANDIDATE]: Term 1 election: Requested vote from peers 3ae6d100de714201a12d52c2a904d4d5 (127.5.122.61:43755), e5f129a86f624ebb904a639e06a83723 (127.5.122.60:35243)
I20260807 03:08:10.923915 6662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e5f129a86f624ebb904a639e06a83723"
I20260807 03:08:10.924446 6662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:10.928141 6662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae92616e623f43e0a0da8f6bfbfc2179 in term 1.
I20260807 03:08:10.928882 6404 leader_election.cc:302] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae92616e623f43e0a0da8f6bfbfc2179, e5f129a86f624ebb904a639e06a83723; no voters:
I20260807 03:08:10.929359 6725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:08:10.940778 6725 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [term 1 LEADER]: Becoming Leader. State: Replica: ae92616e623f43e0a0da8f6bfbfc2179, State: Running, Role: LEADER
I20260807 03:08:10.941406 6725 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:10.948310 6733 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae92616e623f43e0a0da8f6bfbfc2179. Latest consensus state: current_term: 1 leader_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:10.948707 6733 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:10.956559 6742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 03:08:10.961874 6591 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } attempt: 5
Aug 07 03:08:10 dist-test-slave-fgb3 krb5kdc[6357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072090, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:08:10.981078 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.962505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:35243 (local address 127.0.0.1:50128)
0807 03:08:10.962917 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:10.962937 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:10.963071 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:10.963998 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:10.964004 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:10.964576 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:10.964799 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.964813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.966518 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.966525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.968008 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:10.968024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.969265 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.969272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:10.969510 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.970210 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:10.970237 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:10.973289 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:10.978035 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.978041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.978046 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.978622 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.979082 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:10.979087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:10.979092 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:10.979263 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:10.979994 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:10.979999 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:10.980163 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 03:08:10.980318 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 03:08:10.980633 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292}
I20260807 03:08:10.982175 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.963293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.5.122.60:35243)
0807 03:08:10.963478 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.963484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.963504 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:08:10.963570 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.963575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.963680 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.963783 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.965017 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.966261 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.968199 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.968926 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:10.969045 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:10.973537 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:10.973544 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:10.973550 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:10.973592 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:10.977841 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.978768 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.978775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.978779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.978947 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:10.979386 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:10.979392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:10.979400 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:10.979779 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:10.979926 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:10.981537 (+ 1611us) server_negotiation.cc:299] Negotiation successful
0807 03:08:10.981786 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 03:08:10.984364 6407 tablet.cc:2369] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 03:08:10.985647 6636 tablet.cc:2369] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:08:10.988976 6591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5: Bootstrap starting.
I20260807 03:08:10.993328 6591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:10.994978 6591 log.cc:824] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:10.998528 6591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5: No bootstrap required, opened a new log
I20260807 03:08:11.009261 6742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:08:11.034070 6591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:11.038303 6591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae6d100de714201a12d52c2a904d4d5, State: Initialized, Role: FOLLOWER
I20260807 03:08:11.039856 6591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } }
I20260807 03:08:11.041529 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.956059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.5.122.62:37081)
0807 03:08:10.956248 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.956254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.978808 (+ 22554us) server_negotiation.cc:407] Connection header received
0807 03:08:10.979588 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.979595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.979676 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.979744 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.987869 (+ 8125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.988977 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.011501 (+ 22524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.012249 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.012355 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.024970 (+ 12615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:11.024978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:11.024983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:11.025030 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:11.029046 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:11.035873 (+ 6827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:11.035879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:11.035883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:11.036092 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:11.037577 (+ 1485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:11.037583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:11.037586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:11.037977 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:11.038287 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.040773 (+ 2486us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.041056 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 03:08:11.042465 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.958829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45084 (local address 127.5.122.61:43755)
0807 03:08:10.959012 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.959018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:10.978946 (+ 19928us) server_negotiation.cc:407] Connection header received
0807 03:08:10.980029 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:10.980035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:10.980107 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:10.980167 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:10.986503 (+ 6336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:10.987650 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.013292 (+ 25642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.013974 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.014071 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.030758 (+ 16687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:11.030766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:11.030771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:11.030818 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:11.034589 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:11.036949 (+ 2360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:11.036954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:11.036958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:11.037106 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:11.039549 (+ 2443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:11.039554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:11.039558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:11.039959 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:11.040195 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.041804 (+ 1609us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.042057 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 03:08:11.041133 6763 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 03:08:11.046119 6765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:11.047528 6765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:11.051827 6591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:11.052676 6662 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [term 1 FOLLOWER]: Refusing update from remote peer ae92616e623f43e0a0da8f6bfbfc2179: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:08:11.053609 6548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ae6d100de714201a12d52c2a904d4d5"
I20260807 03:08:11.054136 6547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:11.054528 6725 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:08:11.057523 6547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [term 1 FOLLOWER]: Refusing update from remote peer ae92616e623f43e0a0da8f6bfbfc2179: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:08:11.058346 6725 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:08:11.061594 6548 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae92616e623f43e0a0da8f6bfbfc2179 in term 1.
I20260807 03:08:11.046077 6378 master_runner.cc:186] Error getting master registration for 127.5.122.61:43755: OK, instance_id { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" instance_seqno: 1786072090247204 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 03:08:11.066846 6378 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 03:08:11.067806 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:10.990009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50130 (local address 127.5.122.60:35243)
0807 03:08:10.994542 (+ 4533us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:10.994549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.006620 (+ 12071us) server_negotiation.cc:407] Connection header received
0807 03:08:11.009868 (+ 3248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.009875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.009961 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.010039 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.018617 (+ 8578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.019736 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.023775 (+ 4039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.024549 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.024650 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.059656 (+ 35006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:11.059663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:11.059668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:11.059716 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:11.063713 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:11.065063 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:11.065069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:11.065073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:11.065227 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:11.065685 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:11.065691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:11.065694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:11.066053 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:11.066297 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.067138 (+ 841us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.067416 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4391,"thread_start_us":14505,"threads_started":1}
I20260807 03:08:11.093724 6765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae92616e623f43e0a0da8f6bfbfc2179. Latest consensus state: current_term: 1 leader_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:11.094195 6765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:11.115638 6769 mvcc.cc:205] Tried to move back new op lower bound from 7315751284938027008 to 7315751284502446080. Current Snapshot: MvccSnapshot[applied={T|T < 7315751284938027008}]
I20260807 03:08:11.127349 6742 catalog_manager.cc:1347] Generated new cluster ID: b943b3d8540942e39c29985a83ef5673
I20260807 03:08:11.127488 6742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:08:11.144572 6765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:11.144976 6765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:11.144892 6733 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:11.145153 6733 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:11.162268 6702 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae92616e623f43e0a0da8f6bfbfc2179. Latest consensus state: current_term: 1 leader_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:11.162606 6702 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:11.165591 6767 mvcc.cc:205] Tried to move back new op lower bound from 7315751284938027008 to 7315751284502446080. Current Snapshot: MvccSnapshot[applied={T|T < 7315751284938027008}]
I20260807 03:08:11.170192 6702 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:11.170442 6702 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:11.172837 6733 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae92616e623f43e0a0da8f6bfbfc2179" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 37081 } } peers { permanent_uuid: "3ae6d100de714201a12d52c2a904d4d5" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 43755 } } peers { permanent_uuid: "e5f129a86f624ebb904a639e06a83723" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 35243 } } }
I20260807 03:08:11.173203 6733 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:11.182448 6742 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:08:11.184105 6742 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:08:11.200522 6791 catalog_manager.cc:1259] Loaded cluster ID: b943b3d8540942e39c29985a83ef5673
I20260807 03:08:11.200636 6791 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5: loading cluster ID for follower catalog manager: success
I20260807 03:08:11.206187 6742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae92616e623f43e0a0da8f6bfbfc2179: Generated new TSK 0
I20260807 03:08:11.207199 6742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:08:11.214389 6791 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5: acquiring CA information for follower catalog manager: success
I20260807 03:08:11.219864 6791 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ae6d100de714201a12d52c2a904d4d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 03:08:11.376976 6479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.361415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.5.122.62:37081)
0807 03:08:11.361659 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.361665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.374953 (+ 13288us) server_negotiation.cc:407] Connection header received
0807 03:08:11.375278 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.375285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.375378 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.375465 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.376678 (+ 1213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39850: BlockingRecv error: recv got EOF from 127.0.0.1:39850 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 03:08:11.379756 6598 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.361920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.5.122.61:43755)
0807 03:08:11.362115 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.362121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.377934 (+ 15813us) server_negotiation.cc:407] Connection header received
0807 03:08:11.378228 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.378235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.378363 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.378436 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.379521 (+ 1085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45088: BlockingRecv error: recv got EOF from 127.0.0.1:45088 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 03:08:11.381906 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.375160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50138 (local address 127.5.122.60:35243)
0807 03:08:11.375361 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.375368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.380525 (+ 5157us) server_negotiation.cc:407] Connection header received
0807 03:08:11.380650 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.380655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.380736 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.380839 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.381675 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50138: BlockingRecv error: recv got EOF from 127.0.0.1:50138 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 03:08:11.403831 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.396620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.5.122.61:43755)
0807 03:08:11.396794 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.396801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.397095 (+ 294us) server_negotiation.cc:407] Connection header received
0807 03:08:11.397300 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.397305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.397500 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.397649 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.398111 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.399302 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.401151 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.401816 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.401933 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.402517 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.403142 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.403283 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.403420 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.403509 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 03:08:11.413057 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.410907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.5.122.60:35243)
0807 03:08:11.411134 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.411140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.411471 (+ 331us) server_negotiation.cc:407] Connection header received
0807 03:08:11.411768 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.411775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.411898 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.412010 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.412829 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50152: BlockingRecv error: recv got EOF from 127.0.0.1:50152 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 03:08:11.424418 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.418205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45112 (local address 127.5.122.61:43755)
0807 03:08:11.418431 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.418436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.418456 (+ 20us) server_negotiation.cc:407] Connection header received
0807 03:08:11.418524 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.418529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.418632 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.418721 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.419243 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.420372 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.422399 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.423088 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.423218 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.423650 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.423808 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.423896 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.424031 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.424110 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 03:08:11.440474 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.436508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.5.122.60:35243)
0807 03:08:11.436686 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.436692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.438896 (+ 2204us) server_negotiation.cc:407] Connection header received
0807 03:08:11.439178 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.439184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.439276 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.439397 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.440225 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50162: BlockingRecv error: recv got EOF from 127.0.0.1:50162 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 03:08:11.452724 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.445357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45126 (local address 127.5.122.61:43755)
0807 03:08:11.445517 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.445523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.446821 (+ 1298us) server_negotiation.cc:407] Connection header received
0807 03:08:11.446962 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.446968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.447066 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.447166 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.447577 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.448692 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.450570 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.451215 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.451354 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.451846 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.452018 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.452129 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.452313 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.452406 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 03:08:11.468252 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.463667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.5.122.60:35243)
0807 03:08:11.463838 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.463843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.466868 (+ 3025us) server_negotiation.cc:407] Connection header received
0807 03:08:11.467048 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.467054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.467152 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.467269 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.468022 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50170: BlockingRecv error: recv got EOF from 127.0.0.1:50170 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 03:08:11.486426 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.479405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45130 (local address 127.5.122.61:43755)
0807 03:08:11.479624 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.479630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.480121 (+ 491us) server_negotiation.cc:407] Connection header received
0807 03:08:11.480289 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.480295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.480411 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.480516 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.480972 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.482160 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.484155 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.484935 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.485070 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.485574 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.485728 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.485790 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.486023 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.486109 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260807 03:08:11.498733 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.494208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.5.122.60:35243)
0807 03:08:11.494488 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.494495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.497218 (+ 2723us) server_negotiation.cc:407] Connection header received
0807 03:08:11.497396 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.497402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.497510 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.497620 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.498500 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50176: BlockingRecv error: recv got EOF from 127.0.0.1:50176 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
I20260807 03:08:11.516465 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.509562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45142 (local address 127.5.122.61:43755)
0807 03:08:11.509734 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.509740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.510215 (+ 475us) server_negotiation.cc:407] Connection header received
0807 03:08:11.510419 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.510425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.510518 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.510625 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.511061 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.512249 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.514212 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.514981 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.515119 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.515594 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.515769 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.515879 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.516027 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.516117 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 03:08:11.524917 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.523154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.5.122.60:35243)
0807 03:08:11.523330 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.523347 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.523370 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:08:11.523619 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.523625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.523720 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.523832 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.524680 (+ 848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50192: BlockingRecv error: recv got EOF from 127.0.0.1:50192 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 03:08:11.551211 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.544353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45148 (local address 127.5.122.61:43755)
0807 03:08:11.544543 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.544549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.545081 (+ 532us) server_negotiation.cc:407] Connection header received
0807 03:08:11.545275 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.545281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.545374 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.545473 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.545965 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.547205 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.549105 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.549768 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.549901 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.550459 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.550600 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.550698 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.550855 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.550948 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 03:08:11.559902 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.557634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50196 (local address 127.5.122.60:35243)
0807 03:08:11.557838 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.557845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.558386 (+ 541us) server_negotiation.cc:407] Connection header received
0807 03:08:11.558600 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.558606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.558701 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.558800 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.559664 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50196: BlockingRecv error: recv got EOF from 127.0.0.1:50196 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 03:08:11.601040 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.595225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45164 (local address 127.5.122.61:43755)
0807 03:08:11.595457 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.595463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.595802 (+ 339us) server_negotiation.cc:407] Connection header received
0807 03:08:11.595984 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.595988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.596098 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.596173 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.596630 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.597361 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.599264 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.599748 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.599837 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.600349 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.600449 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.600514 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.600674 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.600734 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 03:08:11.608341 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.606511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.5.122.60:35243)
0807 03:08:11.606728 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.606734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.607093 (+ 359us) server_negotiation.cc:407] Connection header received
0807 03:08:11.607230 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.607236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.607332 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.607438 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.608107 (+ 669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50198: BlockingRecv error: recv got EOF from 127.0.0.1:50198 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 03:08:11.680557 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.675490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.5.122.61:43755)
0807 03:08:11.675695 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.675701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.676044 (+ 343us) server_negotiation.cc:407] Connection header received
0807 03:08:11.676230 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.676234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.676306 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.676392 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.676807 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.677525 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.679114 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.679554 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.679647 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.679945 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.680059 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.680123 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.680210 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.680272 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 03:08:11.687383 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.685753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50210 (local address 127.5.122.60:35243)
0807 03:08:11.685950 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.685956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.686169 (+ 213us) server_negotiation.cc:407] Connection header received
0807 03:08:11.686303 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.686308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.686418 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.686520 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.687163 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50210: BlockingRecv error: recv got EOF from 127.0.0.1:50210 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 03:08:11.823448 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:11.818548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.5.122.61:43755)
0807 03:08:11.818775 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.818782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.819069 (+ 287us) server_negotiation.cc:407] Connection header received
0807 03:08:11.819254 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.819258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.819330 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.819409 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:11.819859 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.820551 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.821841 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:11.822248 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:11.822388 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:11.822766 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:11.822889 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:11.822964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:11.823109 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 03:08:11.823188 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 03:08:11.830229 6699 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:11.828694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.5.122.60:35243)
0807 03:08:11.828838 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:11.828842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:11.829140 (+ 298us) server_negotiation.cc:407] Connection header received
0807 03:08:11.829290 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:11.829294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:11.829362 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:11.829434 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:11.830052 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50218: BlockingRecv error: recv got EOF from 127.0.0.1:50218 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 03:08:11.846805 6721 catalog_manager.cc:1259] Loaded cluster ID: b943b3d8540942e39c29985a83ef5673
I20260807 03:08:11.846906 6721 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: loading cluster ID for follower catalog manager: success
I20260807 03:08:11.849293 6721 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: acquiring CA information for follower catalog manager: success
I20260807 03:08:11.851020 6721 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e5f129a86f624ebb904a639e06a83723: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 03:08:12.096757 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:12.091823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45204 (local address 127.5.122.61:43755)
0807 03:08:12.092045 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:12.092052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:12.092314 (+ 262us) server_negotiation.cc:407] Connection header received
0807 03:08:12.092514 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:12.092518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:12.092587 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:12.092662 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:12.093092 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.093835 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.095175 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.095591 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.095698 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:12.096099 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:12.096205 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:12.096279 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:12.096426 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 03:08:12.096508 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 03:08:12.107663 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:12.102172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.5.122.60:35243)
0807 03:08:12.102403 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:12.102409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:12.102512 (+ 103us) server_negotiation.cc:407] Connection header received
0807 03:08:12.102701 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:12.102705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:12.102871 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:12.102973 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:12.103376 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.104101 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.105530 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.105921 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.106048 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:12.106570 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:12.107087 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:12.107231 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:12.107307 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 03:08:12.107398 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 03:08:12.120581 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:12.113500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.5.122.61:43755)
0807 03:08:12.113727 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:12.113733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:12.114705 (+ 972us) server_negotiation.cc:407] Connection header received
0807 03:08:12.115006 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:12.115012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:12.115091 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:12.115184 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:12.115696 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.116703 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.118553 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.119169 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.119292 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:12.119840 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:12.119966 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:12.120055 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:12.120171 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 03:08:12.120252 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 03:08:12.121584 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:12.114732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.5.122.60:35243)
0807 03:08:12.114869 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:12.114874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:12.116027 (+ 1153us) server_negotiation.cc:407] Connection header received
0807 03:08:12.116165 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:12.116168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:12.116240 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:12.116313 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:12.116786 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.117752 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.119761 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.120341 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.120457 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:12.120840 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:12.120955 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:12.121022 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:12.121233 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 03:08:12.121310 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 03:08:12.121724 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:12.113456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.5.122.62:37081)
0807 03:08:12.114103 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:12.114109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:12.114501 (+ 392us) server_negotiation.cc:407] Connection header received
0807 03:08:12.114914 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:12.114920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:12.115156 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:12.115323 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:12.115952 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.116962 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.118970 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:12.119480 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:12.119602 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:12.120668 (+ 1066us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:12.121107 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:12.121200 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:12.121334 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 03:08:12.121418 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":399,"threads_started":1}
I20260807 03:08:12.125771 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6378
I20260807 03:08:12.127022 6468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:08:12.236990 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6378
I20260807 03:08:12.261284 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6486
I20260807 03:08:12.262249 6583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:08:12.365808 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6486
I20260807 03:08:12.389802 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6605
I20260807 03:08:12.390915 6687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:08:12.514097 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6605
2026-08-07T03:08:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4679 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 03:08:12 dist-test-slave-fgb3 krb5kdc[6965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 03:08:12 dist-test-slave-fgb3 krb5kdc[6965](info): set up 2 sockets
Aug 07 03:08:12 dist-test-slave-fgb3 krb5kdc[6965](info): commencing operation
krb5kdc: starting...
W20260807 03:08:13.712514 5608 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/test-user.keytab.
Aug 07 03:08:13 dist-test-slave-fgb3 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T03:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T03:08:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
Entry for principal kudu/127.5.122.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab.
I20260807 03:08:14.007851 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33063
--webserver_interface=127.5.122.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33063,127.5.122.61:33017,127.5.122.60:43175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 03:08:14.345570 6983 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 03:08:14.345873 6983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:14.345952 6983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:14.346036 6983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 03:08:14.353088 6983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:14.353176 6983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:14.353253 6983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:14.353309 6983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:14.353377 6983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:14.362975 6983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33063,127.5.122.61:33017,127.5.122.60:43175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.62.keytab
--principal=kudu/127.5.122.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.62:33063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:14.364076 6983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:14.365284 6983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:14.376507 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:14.376467 6993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:14.376536 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:14.377274 6983 server_base.cc:1034] running on GCE node
I20260807 03:08:14.378010 6983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:14.379361 6983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:14.380594 6983 hybrid_clock.cc:648] HybridClock initialized: now 1786072094380521 us; error 43 us; skew 500 ppm
I20260807 03:08:14.380923 6983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:14 dist-test-slave-fgb3 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:14.388299 6983 init.cc:377] Logged in from keytab as kudu/127.5.122.62@KRBTEST.COM (short username kudu)
I20260807 03:08:14.389752 6983 webserver.cc:460] Webserver started at http://127.5.122.62:40591/ using document root <none> and password file <none>
I20260807 03:08:14.390494 6983 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:14.390609 6983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:14.390944 6983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:14.393949 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data/instance:
uuid: "f41e5aff1c574548aab217c60647c5b6"
format_stamp: "Formatted at 2026-08-07 03:08:14 on dist-test-slave-fgb3"
server_key: "f57f9842d1f752034f1b1f2e81d66ac1"
server_key_iv: "24fbfe9a72ac315fa92f143c8ce4ff5c"
server_key_version: "encryptionkey@0"
I20260807 03:08:14.394800 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/wal/instance:
uuid: "f41e5aff1c574548aab217c60647c5b6"
format_stamp: "Formatted at 2026-08-07 03:08:14 on dist-test-slave-fgb3"
server_key: "f57f9842d1f752034f1b1f2e81d66ac1"
server_key_iv: "24fbfe9a72ac315fa92f143c8ce4ff5c"
server_key_version: "encryptionkey@0"
I20260807 03:08:14.399997 6983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 03:08:14.403410 7002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:14.404680 6983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 03:08:14.404860 6983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/wal
uuid: "f41e5aff1c574548aab217c60647c5b6"
format_stamp: "Formatted at 2026-08-07 03:08:14 on dist-test-slave-fgb3"
server_key: "f57f9842d1f752034f1b1f2e81d66ac1"
server_key_iv: "24fbfe9a72ac315fa92f143c8ce4ff5c"
server_key_version: "encryptionkey@0"
I20260807 03:08:14.405069 6983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:14.426278 6983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:14.430130 6983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:14.430440 6983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:14.448455 6983 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.62:33063
I20260807 03:08:14.448462 7079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.62:33063 every 8 connection(s)
I20260807 03:08:14.449779 6983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/data/info.pb
I20260807 03:08:14.453213 7081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:14.458496 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 6983
I20260807 03:08:14.458657 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-0/wal/instance
I20260807 03:08:14.459072 5608 external_mini_cluster.cc:1428] Setting key df55b268fbdd782965313504abfc40eb
I20260807 03:08:14.459563 7081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:14 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:14.480115 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.462290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33063 (local address 127.0.0.1:41086)
0807 03:08:14.463105 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.463160 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:14.464985 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:14.465759 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:14.465765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:14.466463 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:14.466782 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:14.466797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.468262 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.468266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:14.469338 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:14.469350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.470259 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.470266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:14.470460 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:14.471040 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:14.471061 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:14.473188 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:14.476665 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:14.476675 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:14.476682 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:14.477058 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:14.477489 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:14.477492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:14.477494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:14.477630 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:14.478473 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:14.478478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:14.478650 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 03:08:14.478834 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 03:08:14.479163 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":338,"threads_started":1}
I20260807 03:08:14.480139 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.462291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.5.122.62:33063)
0807 03:08:14.463816 (+ 1525us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:14.463827 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:14.463876 (+ 49us) server_negotiation.cc:407] Connection header received
0807 03:08:14.465318 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:14.465323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:14.465548 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:14.465713 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:14.466968 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.468125 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.469474 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.470050 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.470216 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:14.473418 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:14.473425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:14.473435 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:14.473468 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:14.476547 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:14.477195 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:14.477203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:14.477205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:14.477386 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:14.477744 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:14.477748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:14.477751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:14.478349 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:14.478528 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:14.479009 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 03:08:14.479340 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":1307,"thread_start_us":1112,"threads_started":1}
I20260807 03:08:14.482779 7081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:14.483701 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.483361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:32960)
0807 03:08:14.483520 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.483613 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 03:08:14.483896 7006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
I20260807 03:08:14.485291 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.484970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:32974)
0807 03:08:14.485116 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.485187 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 03:08:14.485461 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:33017: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.122.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab.
Entry for principal kudu/127.5.122.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab.
I20260807 03:08:14.525571 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:33017
--webserver_interface=127.5.122.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab
--principal=kudu/127.5.122.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33063,127.5.122.61:33017,127.5.122.60:43175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:08:14.534754 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } attempt: 1
I20260807 03:08:14.535889 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.535470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:32980)
0807 03:08:14.535654 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.535744 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 03:08:14.537180 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.536774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:32988)
0807 03:08:14.536985 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.537053 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260807 03:08:14.537354 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:33017: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
I20260807 03:08:14.605600 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } attempt: 2
I20260807 03:08:14.606927 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.606304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33000)
0807 03:08:14.606612 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.606734 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 03:08:14.607991 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.607544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33016)
0807 03:08:14.607718 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.607825 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 03:08:14.608174 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:33017: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
I20260807 03:08:14.699451 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } attempt: 3
I20260807 03:08:14.700651 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.700137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33022)
0807 03:08:14.700356 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.700479 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 03:08:14.701646 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.701232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33034)
0807 03:08:14.701388 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.701482 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 03:08:14.701824 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:33017: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
I20260807 03:08:14.845067 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } attempt: 4
W20260807 03:08:14.845067 7092 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 03:08:14.845434 7092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:14.845530 7092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:14.845646 7092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 03:08:14.846294 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.845728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33048)
0807 03:08:14.846018 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.846135 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260807 03:08:14.847319 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.846943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33056)
0807 03:08:14.847080 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.847169 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 03:08:14.847496 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.61:33017: Network error: Client connection negotiation failed: client connection to 127.5.122.61:33017: connect: Connection refused (error 111)
W20260807 03:08:14.853005 7092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:14.853080 7092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:14.853161 7092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:14.853215 7092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:14.853294 7092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:14.862604 7092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33063,127.5.122.61:33017,127.5.122.60:43175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.61.keytab
--principal=kudu/127.5.122.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.61:33017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:14.863673 7092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:14.864830 7092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:14.875408 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:14.875442 7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:14.876060 7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:14.876263 7092 server_base.cc:1034] running on GCE node
I20260807 03:08:14.876953 7092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:14.878186 7092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:14.879391 7092 hybrid_clock.cc:648] HybridClock initialized: now 1786072094879319 us; error 51 us; skew 500 ppm
I20260807 03:08:14.879689 7092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:14 dist-test-slave-fgb3 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:14.882582 7092 init.cc:377] Logged in from keytab as kudu/127.5.122.61@KRBTEST.COM (short username kudu)
I20260807 03:08:14.883831 7092 webserver.cc:460] Webserver started at http://127.5.122.61:38197/ using document root <none> and password file <none>
I20260807 03:08:14.884450 7092 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:14.884546 7092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:14.884824 7092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:14.887666 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data/instance:
uuid: "d6121c0fcb04483096dfec45810d7ddc"
format_stamp: "Formatted at 2026-08-07 03:08:14 on dist-test-slave-fgb3"
server_key: "73497d62a92feee32ce60e01d01806db"
server_key_iv: "8e3240e0bdb7d39f224ebf583cf9d1ca"
server_key_version: "encryptionkey@0"
I20260807 03:08:14.888478 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/wal/instance:
uuid: "d6121c0fcb04483096dfec45810d7ddc"
format_stamp: "Formatted at 2026-08-07 03:08:14 on dist-test-slave-fgb3"
server_key: "73497d62a92feee32ce60e01d01806db"
server_key_iv: "8e3240e0bdb7d39f224ebf583cf9d1ca"
server_key_version: "encryptionkey@0"
I20260807 03:08:14.893266 7092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 03:08:14.896441 7111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:14.897629 7092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 03:08:14.897780 7092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/wal
uuid: "d6121c0fcb04483096dfec45810d7ddc"
format_stamp: "Formatted at 2026-08-07 03:08:14 on dist-test-slave-fgb3"
server_key: "73497d62a92feee32ce60e01d01806db"
server_key_iv: "8e3240e0bdb7d39f224ebf583cf9d1ca"
server_key_version: "encryptionkey@0"
I20260807 03:08:14.897960 7092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:14.919993 7092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:14.923475 7092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:14.923703 7092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:14.939818 7092 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.61:33017
I20260807 03:08:14.939848 7190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.61:33017 every 8 connection(s)
I20260807 03:08:14.940995 7092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/data/info.pb
I20260807 03:08:14.944360 7191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:14.944448 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 7092
I20260807 03:08:14.944617 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-1/wal/instance
I20260807 03:08:14.945029 5608 external_mini_cluster.cc:1428] Setting key 596357488305c4c906cc242bfa322cf1
I20260807 03:08:14.950543 7191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:14 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:14.969424 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.952935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.5.122.62:33063)
0807 03:08:14.953144 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:14.953150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:14.954140 (+ 990us) server_negotiation.cc:407] Connection header received
0807 03:08:14.955970 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:14.955976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:14.956056 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:14.956164 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:14.957711 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.958651 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.960114 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.960532 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.960640 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:14.964143 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:14.964148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:14.964151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:14.964183 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:14.966662 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:14.967431 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:14.967435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:14.967437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:14.967550 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:14.967959 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:14.967962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:14.967964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:14.968206 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:14.968307 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:14.969049 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 03:08:14.969205 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 03:08:14.970636 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.953247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33063 (local address 127.0.0.1:41090)
0807 03:08:14.953979 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.954022 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:14.955704 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:14.956354 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:14.956362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:14.956941 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:14.957547 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:14.957570 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.958808 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.958812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:14.959966 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:14.959979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.960747 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.960755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:14.960968 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:14.961542 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:14.961562 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:14.963943 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:14.966803 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:14.966813 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:14.966829 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:14.967329 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:14.967641 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:14.967645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:14.967648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:14.967869 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:14.968325 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:14.968332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:14.968720 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 03:08:14.968972 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 03:08:14.969427 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":302,"threads_started":1}
I20260807 03:08:14.972191 7191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.122.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.60@KRBTEST.COM" created.
Aug 07 03:08:14 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:08:14.985069 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.972846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33068)
0807 03:08:14.973019 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.973038 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:14.973136 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:14.974036 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:14.974041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:14.974502 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:14.974666 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:14.974677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.976000 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.976005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:14.976878 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:14.976891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.977598 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.977602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:14.977733 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:14.978196 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:14.978214 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:14.980115 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:14.982879 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:14.982884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:14.982888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:14.983190 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:14.983515 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:14.983520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:14.983524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:14.983643 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:14.984182 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:14.984187 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:14.984336 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 03:08:14.984454 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 03:08:14.984703 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 03:08:14.985174 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.973049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.5.122.61:33017)
0807 03:08:14.973662 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:14.973668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:14.973692 (+ 24us) server_negotiation.cc:407] Connection header received
0807 03:08:14.973731 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:14.973736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:14.973881 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:14.973986 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:14.974821 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.975870 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.976990 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:14.977488 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:14.977580 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:14.980293 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:14.980303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:14.980316 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:14.980348 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:14.982770 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:14.983303 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:14.983309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:14.983311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:14.983420 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:14.983736 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:14.983740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:14.983744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:14.984062 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:14.984253 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:14.984616 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 03:08:14.984812 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":343,"threads_started":1}
I20260807 03:08:14.987223 7191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:14.988286 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.987888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43948)
0807 03:08:14.988043 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.988145 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 03:08:14.988534 7115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
I20260807 03:08:14.989913 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:14.989613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43952)
0807 03:08:14.989744 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:14.989816 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 03:08:14.990109 7191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab.
Entry for principal kudu/127.5.122.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab.
I20260807 03:08:15.011662 5608 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
/tmp/dist-test-taskIpVSLO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:43175
--webserver_interface=127.5.122.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab
--principal=kudu/127.5.122.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.20:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.62:33063,127.5.122.61:33017,127.5.122.60:43175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 03:08:15.039470 7191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 1
I20260807 03:08:15.040740 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.040180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43958)
0807 03:08:15.040413 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.040547 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 03:08:15.041716 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.041323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43964)
0807 03:08:15.041477 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.041583 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 03:08:15.041913 7191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
I20260807 03:08:15.110134 7191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 2
I20260807 03:08:15.111325 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.110801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43966)
0807 03:08:15.111010 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.111124 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 03:08:15.112317 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.111948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43980)
0807 03:08:15.112085 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.112169 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 03:08:15.112531 7191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
I20260807 03:08:15.146751 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } attempt: 5
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:08:15.160491 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.147633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33070 (local address 127.5.122.61:33017)
0807 03:08:15.147785 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.147789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.147808 (+ 19us) server_negotiation.cc:407] Connection header received
0807 03:08:15.147870 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.147874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.147956 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.148035 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.149012 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.149679 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.151312 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.151738 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.151825 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.155441 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.155446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.155450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.155479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.157980 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.158813 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.158817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.158819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.158968 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.159358 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.159361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.159363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.159623 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.159730 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.160116 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.160280 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 03:08:15.160691 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.147381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33070)
0807 03:08:15.147600 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.147622 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:15.147755 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:15.148113 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:15.148118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:15.148646 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:15.148872 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.148885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.149812 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.149818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.151168 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.151184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.151884 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.151890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.152118 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.152710 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:15.152733 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:15.155268 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:15.158131 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.158136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.158140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.158702 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.159087 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.159091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.159095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.159258 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.159730 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:15.159735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:15.159900 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 03:08:15.160039 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 03:08:15.160322 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 03:08:15.161655 7081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:15.162639 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.162191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43986)
0807 03:08:15.162386 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.162476 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 03:08:15.163617 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.163262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43988)
0807 03:08:15.163405 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.163487 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 03:08:15.163825 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
I20260807 03:08:15.203760 7191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 3
I20260807 03:08:15.204977 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.204400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:43990)
0807 03:08:15.204646 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.204766 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 03:08:15.206104 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.205605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44002)
0807 03:08:15.205848 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.205950 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260807 03:08:15.206344 7191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
I20260807 03:08:15.215019 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 1
I20260807 03:08:15.216022 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.215521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44006)
0807 03:08:15.215717 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.215817 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 03:08:15.216969 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.216617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44022)
0807 03:08:15.216755 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.216837 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 03:08:15.217187 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
I20260807 03:08:15.285377 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 2
I20260807 03:08:15.286623 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.285981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44038)
0807 03:08:15.286237 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.286422 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 03:08:15.287659 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.287274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44048)
0807 03:08:15.287434 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.287522 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 03:08:15.287896 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
W20260807 03:08:15.345305 7200 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 03:08:15.345615 7200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 03:08:15.345685 7200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 03:08:15.345769 7200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 03:08:15.349581 7191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 4
I20260807 03:08:15.350824 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.350239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44064)
0807 03:08:15.350524 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.350632 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 03:08:15.351787 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.351447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44078)
0807 03:08:15.351581 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.351658 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 03:08:15.352020 7191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
W20260807 03:08:15.353698 7200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 03:08:15.353777 7200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 03:08:15.353866 7200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 03:08:15.353936 7200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 03:08:15.354023 7200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 03:08:15.363948 7200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.20:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.62:33063,127.5.122.61:33017,127.5.122.60:43175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/krb5kdc/kudu_127.5.122.60.keytab
--principal=kudu/127.5.122.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.60:43175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:01:27 UTC on ccycloud.quasar-hkknko.root.comops.site
build id 4571739
ASAN enabled
I20260807 03:08:15.365190 7200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 03:08:15.366532 7200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 03:08:15.377818 7206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:15.377748 7210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 03:08:15.377794 7207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 03:08:15.378628 7200 server_base.cc:1034] running on GCE node
I20260807 03:08:15.379350 7200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 03:08:15.380708 7200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 03:08:15.381915 7200 hybrid_clock.cc:648] HybridClock initialized: now 1786072095381837 us; error 50 us; skew 500 ppm
I20260807 03:08:15.382220 7200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 03:08:15.385237 7200 init.cc:377] Logged in from keytab as kudu/127.5.122.60@KRBTEST.COM (short username kudu)
I20260807 03:08:15.386650 7200 webserver.cc:460] Webserver started at http://127.5.122.60:40885/ using document root <none> and password file <none>
I20260807 03:08:15.387310 7200 fs_manager.cc:359] Metadata directory not provided
I20260807 03:08:15.387404 7200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 03:08:15.387715 7200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 03:08:15.390712 7200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data/instance:
uuid: "57844df64125468b843c731a82964fba"
format_stamp: "Formatted at 2026-08-07 03:08:15 on dist-test-slave-fgb3"
server_key: "1c2dae5ec06ba76c84cfcda388f6b72f"
server_key_iv: "b5fc8f4d2d9306e10c11126dd275b8ec"
server_key_version: "encryptionkey@0"
I20260807 03:08:15.391443 7200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/wal/instance:
uuid: "57844df64125468b843c731a82964fba"
format_stamp: "Formatted at 2026-08-07 03:08:15 on dist-test-slave-fgb3"
server_key: "1c2dae5ec06ba76c84cfcda388f6b72f"
server_key_iv: "b5fc8f4d2d9306e10c11126dd275b8ec"
server_key_version: "encryptionkey@0"
I20260807 03:08:15.394105 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 3
I20260807 03:08:15.395246 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.394743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44086)
0807 03:08:15.394950 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.395049 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 03:08:15.396308 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.395909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44090)
0807 03:08:15.396094 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.396174 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260807 03:08:15.396534 7081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.60:43175: Network error: Client connection negotiation failed: client connection to 127.5.122.60:43175: connect: Connection refused (error 111)
I20260807 03:08:15.396852 7200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 03:08:15.400382 7218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 03:08:15.401638 7200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 03:08:15.401784 7200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data,/tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/wal
uuid: "57844df64125468b843c731a82964fba"
format_stamp: "Formatted at 2026-08-07 03:08:15 on dist-test-slave-fgb3"
server_key: "1c2dae5ec06ba76c84cfcda388f6b72f"
server_key_iv: "b5fc8f4d2d9306e10c11126dd275b8ec"
server_key_version: "encryptionkey@0"
I20260807 03:08:15.401981 7200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 03:08:15.430423 7200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 03:08:15.434778 7200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 03:08:15.435050 7200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 03:08:15.452714 7200 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.60:43175
I20260807 03:08:15.452740 7302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.60:43175 every 8 connection(s)
I20260807 03:08:15.454031 7200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/data/info.pb
I20260807 03:08:15.457686 7303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 03:08:15.461076 5608 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu as pid 7200
I20260807 03:08:15.461238 5608 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIpVSLO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786071733698599-5608-0/minicluster-data/master-2/wal/instance
I20260807 03:08:15.461627 5608 external_mini_cluster.cc:1428] Setting key 36078474ea418d46aee5e789a2dc9d05
I20260807 03:08:15.464849 7303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:15.482928 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.468380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.5.122.62:33063)
0807 03:08:15.468541 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.468545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.469285 (+ 740us) server_negotiation.cc:407] Connection header received
0807 03:08:15.469462 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.469467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.469534 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.469617 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.471032 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.471854 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.473473 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.473893 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.473989 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.477400 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.477405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.477409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.477441 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.480212 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.480934 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.480939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.480942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.481100 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.481430 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.481433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.481439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.481687 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.481794 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.482464 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.482663 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 03:08:15.484227 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.463297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.5.122.60:43175)
0807 03:08:15.464011 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.464022 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.464064 (+ 42us) server_negotiation.cc:407] Connection header received
0807 03:08:15.465726 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.465748 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.466157 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.466541 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.468287 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.470069 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.471632 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.472450 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.472578 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.477026 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.477043 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.477059 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.477103 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.479788 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.480556 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.480564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.480568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.480750 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.481124 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.481127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.481129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.481723 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.482026 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.482297 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.482954 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":517,"spinlock_wait_cycles":19968,"thread_start_us":312,"threads_started":1}
I20260807 03:08:15.484247 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.468451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.62:33063 (local address 127.0.0.1:41102)
0807 03:08:15.469154 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.469189 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:15.469338 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:15.469703 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:15.469710 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:15.470512 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:15.470852 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.470865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.472010 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.472014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.473327 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.473344 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.474131 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.474139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.474352 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.474933 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:15.474969 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:15.477209 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:15.480330 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.480338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.480347 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.480833 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.481197 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.481202 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.481204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.481334 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.481802 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:15.481810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:15.482053 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 03:08:15.482302 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 03:08:15.482845 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":45,"spinlock_wait_cycles":21888,"thread_start_us":343,"threads_started":1}
I20260807 03:08:15.485770 7303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.62@KRBTEST.COM
I20260807 03:08:15.499493 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.486587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.61:33017 (local address 127.0.0.1:33084)
0807 03:08:15.486763 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.486782 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:15.486887 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:15.487189 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:15.487194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:15.487620 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:15.487796 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.487807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.488850 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.488855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.489846 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.489858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.490597 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.490603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.490789 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.491273 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:15.491295 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:15.493548 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:15.496740 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.496746 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.496749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.497140 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.497824 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.497830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.497834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.498036 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.498596 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:15.498600 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:15.498750 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 03:08:15.498872 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 03:08:15.499137 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 03:08:15.499522 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.486592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.5.122.61:33017)
0807 03:08:15.486768 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.486774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.486840 (+ 66us) server_negotiation.cc:407] Connection header received
0807 03:08:15.487001 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.487006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.487068 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.487142 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.487921 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.488711 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.489979 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.490464 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.490555 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.493716 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.493721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.493724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.493754 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.496623 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.497269 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.497274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.497277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.497464 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.498139 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.498145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.498148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.498478 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.498640 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.498983 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.499172 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 03:08:15.500825 7303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260807 03:08:15.501551 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.487666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.5.122.62:33063)
0807 03:08:15.487817 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.487820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.487834 (+ 14us) server_negotiation.cc:407] Connection header received
0807 03:08:15.487953 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.487958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.488030 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.488118 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.489218 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.489898 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.491376 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.491769 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.491857 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.495983 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.495990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.495994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.496031 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.498914 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.499527 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.499532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.499535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.499859 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.500192 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.500197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.500199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.500507 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.500753 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.501041 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.501248 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.60@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.61@KRBTEST.COM
I20260807 03:08:15.514407 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.501586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.5.122.60:43175)
0807 03:08:15.501873 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.501887 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.502057 (+ 170us) server_negotiation.cc:407] Connection header received
0807 03:08:15.502178 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.502181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.502234 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.502304 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.503101 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.503798 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.505375 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.506043 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.506166 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.509278 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.509283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.509286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.509311 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.511924 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.512539 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.512542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.512545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.512660 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.513035 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.513040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.513043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.513339 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.513456 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.513876 (+ 420us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.514120 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":186}
I20260807 03:08:15.514724 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.501643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44120)
0807 03:08:15.501962 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.501979 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:15.502087 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:15.502373 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:15.502378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:15.502750 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:15.502935 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.502947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.503926 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.503931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.505250 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.505262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.506181 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.506186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.506423 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.506926 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:15.506948 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:15.509120 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:15.512035 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.512040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.512044 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.512425 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.512758 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.512763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.512766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.512919 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.513466 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:15.513471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:15.513638 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 03:08:15.513797 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 03:08:15.514298 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":176}
I20260807 03:08:15.517633 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.503094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.5.122.61:33017)
0807 03:08:15.503240 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.503245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.503322 (+ 77us) server_negotiation.cc:407] Connection header received
0807 03:08:15.503479 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.503484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.503544 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.503617 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.505071 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.506012 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.507650 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.508057 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.508175 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.512419 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.512425 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.512429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.512476 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.515369 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.516001 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.516005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.516007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.516136 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.516502 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.516505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.516507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.516743 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.516935 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.517247 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.517412 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 03:08:15.520479 7303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: Bootstrap starting.
I20260807 03:08:15.522403 5608 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 03:08:15.524252 7303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:15.525458 7303 log.cc:824] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:15.528151 7303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: No bootstrap required, opened a new log
I20260807 03:08:15.525298 7036 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f41e5aff1c574548aab217c60647c5b6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41112
I20260807 03:08:15.534916 7303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.535207 7303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57844df64125468b843c731a82964fba, State: Initialized, Role: FOLLOWER
I20260807 03:08:15.535609 7303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.536412 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.536712 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:15.537421 7303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [sys.catalog]: configured and running, proceeding with master startup.
W20260807 03:08:15.546663 7327 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 03:08:15.546810 7327 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 03:08:15.544281 7200 master_runner.cc:186] Error getting master registration for 127.5.122.62:33063: OK, instance_id { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" instance_seqno: 1786072094443359 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 03:08:15.551651 7200 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 03:08:15.573717 7081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 4
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.62@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:08:15.590083 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.574401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44132)
0807 03:08:15.574583 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.574604 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:15.574704 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:15.575053 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:15.575058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:15.575543 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:15.575749 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.575761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.577210 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.577216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.578707 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.578723 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.579665 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.579671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.579866 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.580527 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:15.580551 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:15.583394 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:15.587284 (+ 3890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.587289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.587293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.587809 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.588195 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.588200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.588203 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.588366 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.589067 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:15.589072 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:15.589251 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 03:08:15.589417 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 03:08:15.589703 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 03:08:15.590107 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.574459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.5.122.60:43175)
0807 03:08:15.574652 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.574658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.574681 (+ 23us) server_negotiation.cc:407] Connection header received
0807 03:08:15.574824 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.574829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.574910 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.575025 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.575920 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.577074 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.578854 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.579530 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.579655 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.583562 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.583568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.583571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.583606 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.587149 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.587928 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.587937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.587948 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.588087 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.588479 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.588483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.588487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.588937 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.589088 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.589535 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.589751 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 03:08:15.596238 7081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6: Bootstrap starting.
I20260807 03:08:15.599362 7081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:15.600584 7081 log.cc:824] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:15.603747 7081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6: No bootstrap required, opened a new log
I20260807 03:08:15.608808 7081 raft_consensus.cc:348] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.609107 7081 raft_consensus.cc:729] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41e5aff1c574548aab217c60647c5b6, State: Initialized, Role: FOLLOWER
I20260807 03:08:15.609521 7081 consensus_queue.cc:260] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.610421 7330 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.610773 7330 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:15.611416 7081 sys_catalog.cc:564] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:15.611553 7036 raft_consensus.cc:482] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 03:08:15.611753 7036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:15.615099 7036 raft_consensus.cc:504] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.617290 7261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f41e5aff1c574548aab217c60647c5b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "57844df64125468b843c731a82964fba"
I20260807 03:08:15.617780 7261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:15.621598 7261 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41e5aff1c574548aab217c60647c5b6 in term 1.
I20260807 03:08:15.622220 7004 leader_election.cc:302] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57844df64125468b843c731a82964fba, f41e5aff1c574548aab217c60647c5b6; no voters:
I20260807 03:08:15.622673 7330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 03:08:15.622768 7036 leader_election.cc:288] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [CANDIDATE]: Term 1 election: Requested vote from peers d6121c0fcb04483096dfec45810d7ddc (127.5.122.61:33017), 57844df64125468b843c731a82964fba (127.5.122.60:43175)
I20260807 03:08:15.626261 7330 raft_consensus.cc:686] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [term 1 LEADER]: Becoming Leader. State: Replica: f41e5aff1c574548aab217c60647c5b6, State: Running, Role: LEADER
I20260807 03:08:15.626911 7330 consensus_queue.cc:237] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.651207 7191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } attempt: 5
I20260807 03:08:15.652124 7346 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41e5aff1c574548aab217c60647c5b6. Latest consensus state: current_term: 1 leader_uuid: "f41e5aff1c574548aab217c60647c5b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.652573 7346 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:15.668380 7364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 03:08:15 dist-test-slave-fgb3 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786072094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.61@KRBTEST.COM for kudu/127.5.122.60@KRBTEST.COM
I20260807 03:08:15.677413 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.643411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.5.122.62:33063)
0807 03:08:15.643581 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.643586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.654869 (+ 11283us) server_negotiation.cc:407] Connection header received
0807 03:08:15.655135 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.655141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.655226 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.655295 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.657831 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.659002 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.660982 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.661687 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.661787 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.666418 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.666425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.666430 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.666470 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.670211 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.671511 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.671516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.671519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.671857 (+ 338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.672606 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.672641 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.672645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.673256 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.673693 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.676316 (+ 2623us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.676758 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 03:08:15.678067 7364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 03:08:15.679987 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.645505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.5.122.61:33017)
0807 03:08:15.645674 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.645680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.656334 (+ 10654us) server_negotiation.cc:407] Connection header received
0807 03:08:15.656582 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.656587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.656659 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.656719 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.658883 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.660186 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.664007 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.664595 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.664695 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.670216 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.670223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.670227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.670264 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.675229 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.676353 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.676359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.676362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.676512 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.676935 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.676951 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.676955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.677283 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.677772 (+ 489us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.679403 (+ 1631us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.679636 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 03:08:15.683312 7008 tablet.cc:2369] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:08:15.684751 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.642026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.5.122.60:43175)
0807 03:08:15.642212 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.642218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.648327 (+ 6109us) server_negotiation.cc:407] Connection header received
0807 03:08:15.649453 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.649460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.649583 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.649685 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.652259 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.653453 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.662448 (+ 8995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.663133 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.663228 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.669456 (+ 6228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.669464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.669468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.669508 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.672888 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.681548 (+ 8660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.681553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.681556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.681742 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.682338 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.682345 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.682349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.682722 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.682980 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.683735 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.684149 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 03:08:15.687088 7224 tablet.cc:2369] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 03:08:15.691525 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.651777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.60:43175 (local address 127.0.0.1:44150)
0807 03:08:15.652100 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 03:08:15.652119 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 03:08:15.652223 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 03:08:15.662845 (+ 10622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 03:08:15.662851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 03:08:15.663328 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 03:08:15.663528 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.663540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.667148 (+ 3608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.667154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.668515 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 03:08:15.668531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.669480 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.669486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 03:08:15.669684 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.673134 (+ 3450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 03:08:15.673157 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 03:08:15.675997 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 03:08:15.688757 (+ 12760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.688762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.688766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.689308 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.689723 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 03:08:15.689728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 03:08:15.689731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 03:08:15.689917 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 03:08:15.690557 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 03:08:15.690562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 03:08:15.690711 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 03:08:15.690868 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 03:08:15.691159 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210}
I20260807 03:08:15.691563 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:15.652436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.5.122.60:43175)
0807 03:08:15.662554 (+ 10118us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.662560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.662584 (+ 24us) server_negotiation.cc:407] Connection header received
0807 03:08:15.662631 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.662636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.662706 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.662804 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.665825 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.666990 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.668705 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:15.669312 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:15.675906 (+ 6594us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:15.683285 (+ 7379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 03:08:15.683293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 03:08:15.683297 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 03:08:15.683334 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 03:08:15.688612 (+ 5278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.689416 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.689421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.689424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.689585 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 03:08:15.690068 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 03:08:15.690072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 03:08:15.690075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 03:08:15.690433 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 03:08:15.690559 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:15.690959 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 03:08:15.691152 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9977,"thread_start_us":9796,"threads_started":1}
I20260807 03:08:15.697098 7191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc: Bootstrap starting.
I20260807 03:08:15.697175 7261 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [term 1 FOLLOWER]: Refusing update from remote peer f41e5aff1c574548aab217c60647c5b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:08:15.698189 7346 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:08:15.703337 7191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc: Neither blocks nor log segments found. Creating new log.
I20260807 03:08:15.705438 7191 log.cc:824] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc: Log is configured to *not* fsync() on all Append() calls
I20260807 03:08:15.689481 6983 master_runner.cc:186] Error getting master registration for 127.5.122.61:33017: OK, instance_id { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" instance_seqno: 1786072094935162 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 03:08:15.707739 6983 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 03:08:15.717768 7191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc: No bootstrap required, opened a new log
I20260807 03:08:15.728186 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41e5aff1c574548aab217c60647c5b6. Latest consensus state: current_term: 1 leader_uuid: "f41e5aff1c574548aab217c60647c5b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.728497 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:15.728938 7191 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.729327 7191 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6121c0fcb04483096dfec45810d7ddc, State: Initialized, Role: FOLLOWER
I20260807 03:08:15.729817 7191 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } }
I20260807 03:08:15.731016 7376 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.731681 7376 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:15.732167 7191 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 03:08:15.732200 7154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f41e5aff1c574548aab217c60647c5b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d6121c0fcb04483096dfec45810d7ddc"
I20260807 03:08:15.732553 7153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [term 0 FOLLOWER]: Advancing to term 1
I20260807 03:08:15.737720 7153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [term 1 FOLLOWER]: Refusing update from remote peer f41e5aff1c574548aab217c60647c5b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 03:08:15.740998 7154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f41e5aff1c574548aab217c60647c5b6 in term 1.
I20260807 03:08:15.742769 7372 mvcc.cc:205] Tried to move back new op lower bound from 7315751303929012224 to 7315751303693496320. Current Snapshot: MvccSnapshot[applied={T|T < 7315751303929012224}]
I20260807 03:08:15.743650 7330 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 03:08:15.750651 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f41e5aff1c574548aab217c60647c5b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.750932 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:15.760497 7364 catalog_manager.cc:1347] Generated new cluster ID: 4e146869ae4d41e9b1c989be51913cc3
I20260807 03:08:15.760612 7364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 03:08:15.761101 7330 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f41e5aff1c574548aab217c60647c5b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.761439 7330 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:15.777784 7376 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41e5aff1c574548aab217c60647c5b6. Latest consensus state: current_term: 1 leader_uuid: "f41e5aff1c574548aab217c60647c5b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.778148 7376 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:15.779341 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f41e5aff1c574548aab217c60647c5b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.779938 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 03:08:15.818778 7402 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f41e5aff1c574548aab217c60647c5b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41e5aff1c574548aab217c60647c5b6" member_type: VOTER last_known_addr { host: "127.5.122.62" port: 33063 } } peers { permanent_uuid: "d6121c0fcb04483096dfec45810d7ddc" member_type: VOTER last_known_addr { host: "127.5.122.61" port: 33017 } } peers { permanent_uuid: "57844df64125468b843c731a82964fba" member_type: VOTER last_known_addr { host: "127.5.122.60" port: 43175 } } }
I20260807 03:08:15.819339 7402 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6 [sys.catalog]: This master's current role is: LEADER
I20260807 03:08:15.826013 7364 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 03:08:15.828032 7364 catalog_manager.cc:1504] Loading token signing keys...
I20260807 03:08:15.840420 7364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f41e5aff1c574548aab217c60647c5b6: Generated new TSK 0
I20260807 03:08:15.841516 7364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 03:08:15.850184 7407 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260807 03:08:15.938402 7086 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:15.935776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.5.122.62:33063)
0807 03:08:15.936005 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.936012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.936417 (+ 405us) server_negotiation.cc:407] Connection header received
0807 03:08:15.936749 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.936756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.936868 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.936952 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.938130 (+ 1178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41142: BlockingRecv error: recv got EOF from 127.0.0.1:41142 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 03:08:15.940594 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:15.937038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33110 (local address 127.5.122.61:33017)
0807 03:08:15.937206 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.937211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.939036 (+ 1825us) server_negotiation.cc:407] Connection header received
0807 03:08:15.939276 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.939282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.939401 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.939476 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.940383 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33110: BlockingRecv error: recv got EOF from 127.0.0.1:33110 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 03:08:15.951339 7363 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:15.944358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.5.122.60:43175)
0807 03:08:15.944537 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.944542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.944610 (+ 68us) server_negotiation.cc:407] Connection header received
0807 03:08:15.944770 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.944776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.944873 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.944979 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.951111 (+ 6132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44160: BlockingRecv error: recv got EOF from 127.0.0.1:44160 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 03:08:15.975323 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:15.970576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.5.122.61:33017)
0807 03:08:15.970754 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.970761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.973741 (+ 2980us) server_negotiation.cc:407] Connection header received
0807 03:08:15.973981 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.973987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.974100 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.974203 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.975078 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33120: BlockingRecv error: recv got EOF from 127.0.0.1:33120 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 03:08:15.984253 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:15.979735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33132 (local address 127.5.122.61:33017)
0807 03:08:15.979876 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.979892 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.982867 (+ 2975us) server_negotiation.cc:407] Connection header received
0807 03:08:15.983063 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.983069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.983158 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.983272 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.984049 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33132: BlockingRecv error: recv got EOF from 127.0.0.1:33132 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 03:08:15.991117 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:15.989081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.5.122.61:33017)
0807 03:08:15.989233 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:15.989238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:15.989719 (+ 481us) server_negotiation.cc:407] Connection header received
0807 03:08:15.989889 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:15.989894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:15.989990 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:15.990095 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:15.990886 (+ 791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33146: BlockingRecv error: recv got EOF from 127.0.0.1:33146 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 03:08:16.002455 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:16.000159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.5.122.61:33017)
0807 03:08:16.000355 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:16.000361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:16.000912 (+ 551us) server_negotiation.cc:407] Connection header received
0807 03:08:16.001057 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:16.001062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:16.001150 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:16.001258 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:16.002157 (+ 899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33160: BlockingRecv error: recv got EOF from 127.0.0.1:33160 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 03:08:16.015436 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:16.013321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.5.122.61:33017)
0807 03:08:16.013480 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:16.013486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:16.014031 (+ 545us) server_negotiation.cc:407] Connection header received
0807 03:08:16.014212 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:16.014219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:16.014328 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:16.014437 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:16.015193 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33174: BlockingRecv error: recv got EOF from 127.0.0.1:33174 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 03:08:16.038578 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:16.036222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33186 (local address 127.5.122.61:33017)
0807 03:08:16.036413 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:16.036420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:16.036947 (+ 527us) server_negotiation.cc:407] Connection header received
0807 03:08:16.037144 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:16.037150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:16.037263 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:16.037393 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:16.038295 (+ 902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33186: BlockingRecv error: recv got EOF from 127.0.0.1:33186 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 03:08:16.078804 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:16.076091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.5.122.61:33017)
0807 03:08:16.076283 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:16.076288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:16.076823 (+ 535us) server_negotiation.cc:407] Connection header received
0807 03:08:16.077028 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:16.077035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:16.077139 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:16.077261 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:16.078537 (+ 1276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33188: BlockingRecv error: recv got EOF from 127.0.0.1:33188 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 03:08:16.147440 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:16.145742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.5.122.61:33017)
0807 03:08:16.145924 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:16.145929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:16.146295 (+ 366us) server_negotiation.cc:407] Connection header received
0807 03:08:16.146489 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:16.146493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:16.146567 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:16.146658 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:16.147261 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33200: BlockingRecv error: recv got EOF from 127.0.0.1:33200 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 03:08:16.281327 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:16.279592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.5.122.61:33017)
0807 03:08:16.279818 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:16.279824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:16.280039 (+ 215us) server_negotiation.cc:407] Connection header received
0807 03:08:16.280218 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:16.280224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:16.280338 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:16.280439 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:16.281086 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33216: BlockingRecv error: recv got EOF from 127.0.0.1:33216 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 03:08:16.542830 7197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 03:08:16.540992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.5.122.61:33017)
0807 03:08:16.541205 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:16.541212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:16.541473 (+ 261us) server_negotiation.cc:407] Connection header received
0807 03:08:16.541676 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:16.541681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:16.541754 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:16.541844 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 03:08:16.542611 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33222: BlockingRecv error: recv got EOF from 127.0.0.1:33222 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 03:08:16.548635 7327 catalog_manager.cc:1259] Loaded cluster ID: 4e146869ae4d41e9b1c989be51913cc3
I20260807 03:08:16.548729 7327 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: loading cluster ID for follower catalog manager: success
I20260807 03:08:16.551165 7327 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: acquiring CA information for follower catalog manager: success
I20260807 03:08:16.552826 7327 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 57844df64125468b843c731a82964fba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 03:08:16.854280 7407 catalog_manager.cc:1259] Loaded cluster ID: 4e146869ae4d41e9b1c989be51913cc3
I20260807 03:08:16.854427 7407 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc: loading cluster ID for follower catalog manager: success
I20260807 03:08:16.856899 7407 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc: acquiring CA information for follower catalog manager: success
I20260807 03:08:16.858701 7407 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6121c0fcb04483096dfec45810d7ddc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 03:08:17.066879 7505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:17.060166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.5.122.61:33017)
0807 03:08:17.060924 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:17.060929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:17.060958 (+ 29us) server_negotiation.cc:407] Connection header received
0807 03:08:17.061021 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:17.061025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:17.061160 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:17.061268 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:17.061782 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.062668 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.064489 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.064910 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.065009 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:17.065620 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:17.066168 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:17.066275 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:17.066451 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 03:08:17.066521 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":478,"threads_started":1}
I20260807 03:08:17.079227 7517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:17.072719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.5.122.60:43175)
0807 03:08:17.073208 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:17.073214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:17.073533 (+ 319us) server_negotiation.cc:407] Connection header received
0807 03:08:17.073722 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:17.073726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:17.073873 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:17.074012 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:17.074439 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.075188 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.077013 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.077404 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.077510 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:17.078097 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:17.078641 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:17.078745 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:17.078879 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 03:08:17.078946 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260807 03:08:17.106488 7529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:17.085135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.5.122.62:33063)
0807 03:08:17.096229 (+ 11094us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:17.096236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:17.097205 (+ 969us) server_negotiation.cc:407] Connection header received
0807 03:08:17.097355 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:17.097362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:17.097604 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:17.097794 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:17.098731 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.099936 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.103342 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.103964 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.104089 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:17.104837 (+ 748us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:17.105331 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:17.105419 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:17.106052 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 03:08:17.106146 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10946,"thread_start_us":11936,"threads_started":1}
I20260807 03:08:17.108074 7517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:17.085496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.5.122.60:43175)
0807 03:08:17.085647 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:17.085653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:17.100658 (+ 15005us) server_negotiation.cc:407] Connection header received
0807 03:08:17.101027 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:17.101033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:17.101154 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:17.101238 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:17.102101 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.103163 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.105240 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.105860 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.105958 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:17.106717 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:17.106861 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:17.106923 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:17.107706 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 03:08:17.107790 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 03:08:17.110666 7505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 03:08:17.091541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.5.122.61:33017)
0807 03:08:17.091719 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 03:08:17.091724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 03:08:17.103188 (+ 11464us) server_negotiation.cc:407] Connection header received
0807 03:08:17.103331 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 03:08:17.103337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 03:08:17.103429 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 03:08:17.103543 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 03:08:17.104438 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.105592 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.108125 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 03:08:17.108856 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 03:08:17.108979 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 03:08:17.109914 (+ 935us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 03:08:17.110030 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 03:08:17.110114 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 03:08:17.110253 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 03:08:17.110370 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 03:08:17.111372 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 6983
I20260807 03:08:17.112310 7073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:08:17.222666 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 6983
W20260807 03:08:17.248449 7112 connection.cc:535] server connection from 127.0.0.1:33070 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 03:08:17.248848 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7092
I20260807 03:08:17.249889 7183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:08:17.354378 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 7092
I20260807 03:08:17.377805 5608 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu pid 7200
I20260807 03:08:17.378893 7293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 03:08:17.483932 5608 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIpVSLO/build/asan/bin/kudu with pid 7200
2026-08-07T03:08:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4969 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9649 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (363706 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST